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MEMORANDUM FOR DEPUTY SECRETARY OF DEFENSE (ACQUISITION REFORM) 

FROM: ACTING ADUSD(AR/AP&P) 

Prepared by Lt Col Floyd, 693-7794. June 17. 1996 

SUBJECT: Past Performance Information Study 
SUMMARY: 

• On June 12. 1996 Arthur D. Uttle delivered a review draft of the final report for subject 
study. Feedback was provided to the contractor by Lt Col Floyd on June 14, 1996. The 
revised final report, dated June 17, 1996 is attached. 

• Pages 82 through 91 contain ADUs conclusions and lessons learned. 

• In response to DUSD(AR) specific questions the report concludes the following: 

- Should Dob use past performance? Yes. because: 

(1) it makes good business sense - has overwhelming industry acceptance. 

(2) it is being used successfully in DoD now, although on a limited scale. 

(3) it can be tailored - it should be made clear who may do tailoring and to what 
extent. 

- What information should be collected--what type of approach should be used and what 
direction and guidance should be provided? 

(1) Decentralized approach supported by general guidelines, decision njles, best 
practices,' and information technology support. 

(2) Business area focus. 

(3) Total program context 

(4) Horizontally integrated business areas across DoD. 

(5) User-driven - including technical, management and procurement. 

(6) Share of PP infonnation across DoD should be considered after above 
considerations. 

(7) PP approach needs to be simple and comprehensible. 

(8) PP policy implementation should follow the tenets, procedures and techniques of 
Contractor Evaluation Program ("To-be model"). 

(9) FAR Part 42 PP infonnation collection requirements: 

- Implemented for commodities, except commercial items. 

- Tested on pilot/prototype basis for sen/ices. 

- Not be required for major/small systems - should be tested on pilot/prototype 
basis with emphasis on evaluation of processes. 

• DFARS Case 95-D715 was forwarded to DAR Council June 14, 1996 for publication in the 
Federal Register as a final mie. 

• We now need to formulate a "roll-ouf plan for the study findings which would include 
implementation recommendations and call for plans from DoD components. 
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Section 1: Executive Overview 



A. Background 



PoUcy, legisladoo. reguladons. and guidanc have been issued on con»«or pa« 
perfomiance as it relatBto Government contracting. 

This acuon has c».sed son« concern within ^^^^^'^^Zt Defense 
implemenudon. Because of the»com=emste^^^ 



performance. 



B Objective and Scope 

use of information on the past performance of contractors. 



C. Summary of Results 

Existing 000 and other Governn«nt Agency Pas. Pertonnanc. Proc^ses and 

Systems 

. Performance infonnadon gathered on a routine basis as contraa work « 
performed, which is further divided into: 



. infonnaaon gathered for pmposes of ma«agmg U« active contr^^^^ 
. Information gaU.e«d witi. *e in^ntion tt«t it will be made available for use 
in acquisition decisions at some future date. 

T ,i„„™« of identifying and evaluating existing past performance information 
the existing processes and systems: 

. Information is collected, validated and med for the specific purpose of 

supporting future source selection decisions, . , k 

. Opportunityisprovidedforreview.commentandrebuttalofthemfo^^^ 

the contractor, 

. Provisions are established to n«olve disputes between the contractor and the 

government concerning the vaBdity of the informauon. 
. Lormattonissubj«:ttod.esamecontrolsandsafeguardsasofl.ermformauon 

used in source selecfiondecbions, and ' . . 

. The system is in operation and cuttentiy supporting source selection decstons. 

■n« appUcation of titese definitions and criteHa revealed a vcn- Umi«d ~ J"^*' 

p'^r^rformanoe and which draws on information that may be available from tite two 
otiier types of systems noted above. 

Tased on the fact tiiat award fee contracts cover about 80% of NASA s procureme 



doUars and d«se contacts al«ady provide for a P"'"^^^,'^^^^ , 

as a standard evaluation factor in source selecUon. 
Industry Supplier Evaluation Programs 

A„„*=r Glared Knd that was evident in nuny todnstries was a move to «ablish «»ore 
Although the todustry progran. varied •^^rl^^o^'^-^'^ 

rrd^™ri=^^.^2--^^ 

business area with regard to company requuenjents. Pf^^f^P^J^^^^^, 
the details of the specific business area. 
Contractor Evaluation Program Model 

relationships and performance with contractors and suppUers. 
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Consequently, as a part of the study effort a Contractor Evaluation Program model was 
drvd^iTto ^^e L a &ame of reference for evolving a consensus on a viab e 

.^not^nlv for dealing with contractor past performance issues, but also for 
:55r^g r^^^d Ldicated above-Lket analysis, best value contractmg. 
and world class contractors and suppliers. 

This model was developed from the Aurthur D. Little case histories and discus^ons with 
ItwTlge o1 Government and industry officials who reprinted a -os-s^^- 
tZ L^ms and discipUnes that are involved in the acqmsition of products and 
^r^ T^dition. L workshops were held with representatives of Acquisition 
Reform Scenarios Steering Group (ARSSG) members. 

SZiT s^p in achieving the objectives of the study. As currently env«.oned, the 

Contiactor Evaluation Program model has three principal elenKnts: 

. Business area pla«-an analysis of the business areas in which contracts are 

awarded 

. Business area strategy-an approach for dealing with contractor past performance 

and related issues in each business area 
. Business area evalution process-the process for executing the approach m each 

business area. 



The principles used in designing the model included the foUowing: 



A cost-effective approach to the coUection and use of contractor past 
trfo^ce infoi^Lion depends on, and is sensitive to factor rcl^f^^^^ 
SS^Las in which products and services are procured and °PP° 
Ta Jversal approach L can be appUed to the M range of products and 
services procured by DOD in all sectois of the mdustry). 

A business area consists of a homogeneous group of products or servu:es wluch 
fhSSl^Tharacteristics and for wMch a forwardjo^^^ 
and congruous strategy and evaluation process can be developed. 

, BusinessareascanbelocalorextendedinappUcation^fathe^m^^^^^ 
form, they constitute the horizontal integration of products and services. 

, TTieprocessforimplementingcontractorpastperformanceissu^^^^ 

business area is developed from business ar^ plans and strategy for the specific 
business area and typically involves a cross-functional team effort 

. Tl.e initial and vital step in developing plans and strategy ^^^^"^^ 
In analysis that covers the requirements for die product or ^^^^f"^^ 
Ejected; tiie industry composition and basis of competiUon; an^^^ 
S ani specific performance of leading compames m die mdustry. 

. nie business area plan and strategy will provide die basis for devel^S ^ 

performance. . • u ^ ^« 

SrSess aref and plans as weU as the pas. performance of 

individual contractors in diose busmess areas. 

A description of die Contractor Evaluation Program is contained in Section ffl. C. of the 
report 



Business Case Analysis 

The analyses we,e conducted ftom econonucs automated data infonnaiton systems, and 
comparison perspectives. 

The business case analysis focused on the following areas: 

. Process and automated systems analysis of current past performance systems; 



5 



. AnalysisofrecentchangestotheFARandproposedchangestotheDF^^ 

contractor past perforarance relative to the As-Is model 
. A proposed approach for dealing with contractor past perfomrance issues, 

referred to as the 'To-Be Model"; and 
. Differences between the porposed model and the FAR/DFARS approach. 

collecting the information. 

We reviewed and diagrammed the process for the systems that are currently in i^e. and 
Intified .Te principd activities that are involved in the coUectjon -^^^ ^fj^ 
^ormance iSfonr^tion. Hiese process analyses are discussed m the body of the 

report 

examining the existing systems, the major cost elemen^ were '° 
Se ^Son and vaUdation of performance information for possible future use m 
source selection decisions. The major steps in that process are: 
. Opportunity provided for contractor to review performance information 
. Response possibly provided by the contractor in the form of comments, rebuttal. 

or additional informaiton 
. Any response from contractor reviewed by Government officials and decisions 

made on possible adjustment to the evaluation 
. All information treated as "Source Selection Informaiton" and fded for possible 

future use for a three-year period. 

Attributing specific and quantifiable benefits to the existing systetns was found to be 
e,!Smely difficult Facets that compUcated these detemunanons mduded the 

following: ' a 

. Some of the information systems were in the early st^es of implementation and 

specific, tangible benefits were yet to be demonstrated; 
. The value of specific information was difficult to isolate because of the muluple 

sources and types of information that are available for use m a procurement 

rr^rs.ld^ause the ultimate award decision ty^^^^ 

information from many sources; and 
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. The existing systtms were <aao«d to ^^"^J^^"^, ,«hich 
infonmtioi. and evaluation factors unique to the pamcular busmess area (wlucl. 
«d not necessarily have relevance outside of tiat busmess area). 

A svstem used by the Air Force for m^or progn«ns, as well as the ^^"f «« 
^S^2ntcr archiUKX engineering and consHuoUon work, are .adored to 

ScS^ogo-^ to busing areos. In so doing, evaluafon fa«o^ 
d^eto^d used fn these systen^ that had direct relevant to the qrpe of woric 

typicaUy contracted for in die particular busmess area. 

Whereas dKSe systems seem to be operating weU and tor the purpose intended. Aere 
~cll.^t an attempt to design a s^ »~ver^^^^^^ 

body of the report 

Because of the design features of the Contractor Evaluation . 
n^lv enumer^. the cost/benefit ratio appears to be very favorable. Th^ ^ 

JTare formed in organizations with direct acquisiuon responsibility. 

the appUcation and - 'j™ ^StT^r^'c^o^ and to describe 
Sr;:nL"&Te;:;SdrLappear reasonable and f^^^^^ 



point. 



fh^tZrre criteria that could be used to eval^^ „ 



information system that prescribes the same detaUed evaluation criteria and common 

enSL in evaluating contractor performance in all acquisition cases and 
proposes to coUect that information in one system. 

The business case analysis is in Section ffl. D. of the report 



D. Conclusions 

Question: Should DOD use past performance? 

Answer Yes, because: 

• it makes good business sense 

• it is required by law and regulation ^ u u 
. it can be tailored to fit specific circumstances, although it is not clear who should 

do the tailoring and to what extent 

Question: What information should be coUected-what type of approach should be 
used and what direction and guidance should be provided. 

Answer The DOD approach should fbllow these general principles: 

. Decentralized-The range of products and services ^nd *e j^^^^^^^^ 

scope type, and complexity of contracts makes a standard, DOD-wide system 
mp^S Government and industry experience suppoit a decentralized 
appr^ch supported by general guideUnes, decision rules, best practices, and 
information technology support 
' . Focused on Business Areas-Tlie implementation of past performance should 
focus on individual business areas at the operating level that encompass similar 
products or services for which a coherent and congruous strate©^ can be 
developed by organizations with procurement authonty and techmcal 
responsibility. 

. Total Program Context-Past performance needs to be viewed in thecontext of a 
total pro^un that goes beyond the coUection and use of past performance 
information, and covers: 

• Analysis of individual business areas, to include both internal and 
external factors 



. Development of a sensible strategy for contractor past perfoni^^^ 

business area level. . u - u 

. Processes designed to implement the strategy for business areas m which 
the organization is active 
Horizontally Integrated-The business area concept starts at the local level 

S.^^ts3?)LA. TlSmplem»d.g direcdon „«* » .mphas,z= *e 
for this integration and coordination. 

in and responsible for making contractor selecuon decisions, 
with after all of the above are dealt with. 

Simple-To be cffecdvc. U,e past perfonnan« ^roa^h tas » be e^y » 
undersmd and explain, without being simphsuc, or « .^s the nsk of being 
misunderstood, ignored, or both. 



9 



Section II. Goals, Objectives, and Methodology 



A. Background 

Policy, legislation, regulation, and guidance have been issued over the past tiuee years 
on nuitters related to the collection and use of contractor past performance informauon 
in the awarding of government contracts. 

Methods and approaches for implementing this direction and gui^ce within the DOD 
have been studied, discussed, and evaluated by officials in the Office of the Secretary of 
Defense (OSD), in each of the military services, and in the Defense Logisucs Agency 
(DLA) In addition, the Past Performance Coordinating CouncU (PPCC) Prof^"^^^"^ 
reprei^ves from the services and DLA have been active in developing a DOD-wide 
position on contractor past performance. 

The current plan in DOD is to issue a change to the Departinent of Defense FAR 
Supplement (DFARS) estabUshing tiie manner in which contractor past performance 
wiU be handled witiiin tiie Department of Defense. The DFARS change is m the final 
stages of coordination. 



In addition, there has been, and continues to be, some concern and reservation on the 
part of officials within tiie DOD on many of die issues tiiat surround tiie implementauon 
of tiie existing poUcy and regulations-as well as the contemplated changes tiiat are 
contained in tiie draft DFARS. Altiiough die poUcy and regulations provide that 
implementation can be tailored to tiie particular circumstances and nature of a 
procurement program, tiiere has been no consensus on tiie extent of tiie tailonng that 
should or could be done, nor on who should be empowered to do the tailonng. 

Because of tiiese issues and concerns, tiie Deputy Under Secretary of Defense 
(Acquisition Reform) determined tiiat further study was needed before implementing a 
department-wide policy dealing witii matters related to contractor past performance. 
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B. Objective and Scope 

Tu nhi*.ctive of the study was to provide infonnation and independent 

IvlSt^htmi^^^^^ 

and use of infonnation legarding the past performance of contractois. More 

specifically, the study was twofold. First, it was to consider. 

. AU existing past performance processes and systems used within the DOD, and a 

slptem^le processes and sys 

commercial firms; 

. The manner in which past performance information is coUected and vahdated; 

. Hie past performance data elements prescribed by the functional users; 

. Customer satisfaction with past performance information and the systems that 

provide this information; 
• Customer views on the difference that past performance information makes m 

the source selection process; 
. An economic analysis to determine whether the use of past performance 

information makes good busmess sense; 
. The appropriate use of contractor past performance mformation; 
. The current use of past performance information within DOD; and 
. The administrative burden associated with coUecting the information. 

Second, it was to develop a model program, using the results of the above 
considerations, to assist past performance poUcy implementation. 



C. Methodology 

The principal steps in our approach for this study were: 

• Research 

• Review 

• Interviewis 

• Analysis 

• Benchmarking 

• Model Program Development 

• Functional Requirements Development 
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• Business Case Analysis 



A descriotion of the major activities that occurred as these steps were applied and the 



each step follows 



Research 



areas 



History of past performance information systems in DOD. We exanuned the 
W^o^of other past performance efforts in DODsmce the 1960s. The 

coSon and uL of past performance information in source selection decisions 
Ltrn^!^u^usV-aches have been trie^ and this historical perspective 
has proven useful in guiding the study as well as developmg our 
recommendations. 

PoUcy, Legislation, Regulations, and Guidance. 
The principal documents reviewed included: 

. OFPP's Policy Letter 92-5, Past Performance Information, December 30, 
1992 

. Section 1091 of the Federal Acquisition Streamlining Act of 1994 (FAS A) 
. Federal Register, March 31, 1995. Federal Acquisition Regulation, Past 

Performance hiformation. Final Rule 
. OFPP's "A Guide to Best Practices for Past Performance," Interim Edition, 

May 1995. o , , 

. Proposed amendment to Defense Federal Acquisition Regulation Supplement 

(DFARS) 

. DUSD(AR) and Past Perfomrance Coordinating CouncU information 

Arthur D. LitUe's Industry Supply Chain Management Database. Arthur JD^ 
LitUe maintains a database of Industiy Supply Cham Management case histones 
which was reviewed. 



12 



Reviews of Past Performance Processes and Systems 

A comprehensive review process was used to identify and categorize existing past 
performance information systems in the federal sector. The approach we foUowed was 
to first identify all systems and processes that contained, or in some way dealt with, past 
performance information. They are as follows: 



Systems and Processes Related to Past Parfonnance 





Acronym 




Owner 


1 


ABVM 


Automated Best Value Model , 


DLA 


2 


ACASS 


A&E Contract Administration Support System 


COE 


3 


ACPS 


Automated Contract Preparation System . . 


Air Force 


4 


ACTS 


Automated Configuration Tracking System 


DCMC 


5 


AMIS 


Acquisition Mana^mcnt Information System 


Air Force 


6 


BCAS 


Base ContractinR Automation System 


Air Force 


7 


BRP 


Blue Ribbon Program 


All DOD 


8 


assR 


Cost/Sciieduie Status Reports 


DOD 


9 


CCASS 


Construction Contract Appraisal Support System 


COE 


10 


cess 


Commodity Conunand Standard System 


Army 


11 


CDCS 


Customer E>epot Complaint System ^ 


DLA 


12 


CIS 


Contractor Information System 


Army 


13 


CIS 


Contractor Information Service , 


DCMC 


14 


CPARS 


Contract Performance Assessment Reporting System 


Air Force 


15 


CPR 


Cost Performance Reports , — 


DOD 
DCMC 


16 . 
17 


CPS 
DPACS 


Contractor Profde System 

DLA Pre-award Contracting System 


DLA 


18 


GIDEP Alerts 


Government Industry Data Exchange Program Alerts/5ate 
Alerts ' 


DOD 

Air Force 


19 
20 

21 


J041 

JACG-IPT 

MIR 


Acquisition and Due In System 

Joint Aeronautical Commanders Croup Integrated Product Team 
(study covers contractor past performance and supplier rating) 
Material Inspection Records ^ 


Joint Service/ DLA 
Navy 


22 
23 


MOCAS 
PADDS 


Mechanization of Contract Administrative Services 
Procurement Automated Data and Document System 


DCMC 
Array 


24 


PASS 


Pre-award Survey System 


DCMC 


25 


PDREP 


Product Deficiency Reporting and Evaluation Program 


Navy 


26 


POOR 


Product Qualitv Deficiency Reports 


DCMC 


27 


PRAG 


Performance Risk Assessment Groups 


Armv/ AF 


28 


PROCAS 


Process Oriented Contract Administration Services 


DCMC 


29 


pPL 


Qualified Parts List 


Navy 


30 


RAM 


Risk Assessment Model 


DCMC 


31 


RYG 


Red Yellow Green 


Naw 


32 


SAACS 


Standard Arniy Automated Contracting System 


Armv 


33 


SALT 


Svstem Analvsis and Lab Testing 


DLA 


34 


SAMMS 


Standard Automated Material Management System 


DLA 


35 


VRS 




Air Force 



In the next step, we used the definition of past performance information to focus on the 
more relevant systems and processes. According to the OFPP Policy, past performance 
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information regarding a contractors actions under previously awarded contracts is 
relevant information. Past performance information includes the contractor's: 
. Record of conforming to specifications and to standards of good workmanship; 

• The contractor's record of containing and forecasting costs on any previously 
performed cost reimbursable contracts; 

• Adherence to contract schedules, including the administrative aspects of 
performance; 

• History for reasonable and cooperative behavior and commitment to customer 
satisfaction; and 

• Business-like concem for the interest of the customer. 

The next step involved further sciwning of the systems and processes using the 
definition for past performance information systems. This definition was denved from 
guidance and direction contained in OFPP PoUcy Letter No. 92.5, FAR chants (FAC 
90-26), and OFPP "A Guide to Best Practices for Past Performance." The defimtion 
used for past performance information systems is as follows: 

• Information is coUected,vaUdated and filed for die specific purpose of 
supporting fiitiire source selection decisions; 

• Opportunity is provided for review, comment and rebuttal of die information by 
the contractor, 

• Provisions arc established to resolye disputes between the contractor and tiie 
govemment; 

• Information is subject to the same controls and safeguards as otiier information 
used in source selection decisions; and 

• System is in operation and currently supporting source selection decisions. 



Interviews 

We conducted interviews to obtain current information on existing past performance 
processes and systems. The chart below illustrates the approach we used to gather data 
on existing past performance process and systems. 



14 



Arthur D. Little SuppV 
Chain Management 
Commercial Database 



ODUSD(AR). OLA. 
SERVICES. PPCC 
Information 



Task 2 Criteria 



FASA. FAR. DFAR 
Guidance 



information needed for 
Task 4.5. &6 




Industry 

• OODH'eiatBd 

• No/Minimum DOD 
involvement 



DOD 

• OSO, Servfoes. 
Agencies 



ll 

] 



Other Government 
• NASA, DOE. GSA 



Data to construct the interview guides was obtained from Arthur D. -^"PP^J^^" 
basement databases; information provided by DUSD (AR); the evaluation catena 
TtS ii ^estatetr^nt of work for the study provided a^^^^^ 
Federal Acquisition Streamlining Act. Federal Acquisition Regulation, and Office of 
FeS pS:urement PoUcy documents; and questions for information we anticipated 
would be needed later in the study. 

The interview guides and their appUcation provided a consistent, structured approach to 
^^^o^M indicated in right-hand side of the above chat n.~ were 
conducted in three major sectors-DOD. including OSD, the Services and DLA. 
industrv including both defense airf commercial contractors; and non-DOU 
Oov«t ag«.cies. including the Nadonal Aeronautics and Space Adnumstration 

o?En.rg,. Department of Comme,«. General Services Admmtstrauon. and 
Department of Transportation. 

An important outcome of this approach was the broad industry and government response 
we were able to obtain. Such response was significant in that it assured broad 
Z^Z^l and helped mitigate potential bias. A wide range of ^OD o^^-^^^^^^ 
wL contacted for mformation relevant to this study. Interviews were conducted with 
representatives of: Defense Logistics Agency (DLA) at "eadqu^e^ and at t^^^^^^ 
Defense General Supply Center; Defense Contract Management Command As istant 
Secretary of the Army/Research, Development, and Acquisiuon; Army Matenal 
ComiZd; Army Corps of Engineers; Secretary of the Air ^^^^cq^^^^^^ 
Contracting; Air Force Material Command/Procurement and Wnght Laboratories 
Assistant Secretary of the Navy/Research Development ^'^^XZ^ol vt^cr^i 
Systems Command; and Naval Material Quality Assurance Offi<j. Non-DOD Federal 
Agendes were also contacted. GSA in particular provided significant coverage m terms 
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of contracting experience since GSA manages many contracts that serve other federal 
govemment agencies. NASA provided a technology perspective and DOE the 
perspective of complexity. Such wide organizational representation also provided a 
diversity of experience in terms of the nature of the products and services that were 
acquired. 

It is important to note that in conducting the interviews^ information was gathered not 
only from users of past performance systems and processes, but also from the managers 
and owners of such systems and processes. This approach provided assessments and 
ideas from many individuals representing the different points of view in the process. 
Together, the DOD and non-DOD sources of information provided a relatively large 
experience base to draw upon. This base was a particular strength of this study. The 
intervie>y guides are included in Appendix A. 



Analysis 

Analyses were conducted for each of the existing systems to determine the success of 
each system in meeting its past performance information system objectives. Other 
analyses examined the extent of coverage provided by the existing systems relative to 
contract dollar value, product or service areas, and the evaluation factors cited in the 
\ OFPP guide on Contractor Past Performance. 

In addition, each of the systems and processes was compared to the evaluation factors 
contained in the Statement of Work. These factors covered the following: 

• Data System Design-centralized or non-centralized 

• Kinds of data used— government, private 

• Integrity of data— identity of sources 

• Accuracy 

• Currency 

• Remedial Process by Contractors 

• Availability of Information for Source Selection 

• Confidentiality 

• Sub-contractor Involvement 

• Maintaining Identity of Contractors That are Acquired 

• Fairness 

• Due Process 

• Lackof Past Performance 
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• Threshold of Applicability 

• Capability of Attribution 

• Penalty 

Each of the services and DLA have initiatives underway v^rhich aim at expanding the 
past performance information avaUable for use in contractor selection decisions. 
Planned past performance information sources were identified m the study along with 
the conceptual approach that wiU be applied and the depth of coverage. 

Systems used by other government agencies were also analyzed, focusing on the 
following four areas: 

• Published Policies 

• Rating System 

• Databases 

. Known problems with existing approach to past performance evaluation 



Benchmarking 

The benchmarking phase of the study was developed through on-site visits and the 
review of information in the Arthur D. Little Supply Chain Management practice 
database and secondary research. For benchmarking purposes we interviewed 
companies which are considered to be best in class in terms of suppher past performance 
evaluation. These firms included: 
Allen-Bradley 
Baxter Health Care 
Black & Decker 

Boeing Defense and Space Group 
Fisher Scientific 
Ford Motor Co. (by telephone) 
McCormick & Co. 
McDonnell Douglas/C-17 
Mobile Corporation 
National Semiconductor 
Rockwell North American 
U.S. Postal Service 
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• W.W.Grainger 

We compUed the results into a series of "best practices" for DUSD(AR) consideration. 



Model Program Development 

The Contractor Evaluation Program is the model program we developed to assist 
DUSD(AR) in past performance poUcy implementation. We approached the 
development of the Contractor Evaluation Program by refining our benchmarking 
results and by conducting workshops for DOD officials who were involved in 
acquisition reform initiatives and who represented the functional areas tiiat were 
affected in some way by contractor past performance processes and systems. 

In so doing, several workshops were held witii tiie foUowing representatives from die 
Acquisition Reform Senior Steering Group (ARSSG) and tiie Past Performance 
Coordinating Committee (PPCQ: 
Major Programs (APD 
Logistics 

Economic Security 
Systems Engineering 
Quality 

Inspector General 
Procurement 
General Counsel 

Defense Contract Management Command (DCMC) 
Defense Contract Audit Agency 

Workshops were built around four inter-related modules, listed below: 

• Review background informaiton (address new poUcies; government and industry 
programs) 

• Develop working definition of contractor past performance 

• Assess selected contractor past performance evaluation practices 

• Develop a working process for contractor past performance 
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Workshop participants were introduced to the goals, objectives, and desired outcome of 
the study. In order to provide a baseline for each workshop, information was provided 
to participants on DOD 5000 and FARyDFARS, as well as on conunon elements 
associated with a contractor evaluation program. In addition, industry supplier 
evaluation programs and lessons learned from industry were shared with participants. 

During the workshops, maximum opportunity was provided for participants to share 
their perspectives on past performance evaluation. Participants also addressed questions 
concerning a DOD contractor vision and implications of anticipated changes for the 
acquisition community. 



Functional Requirements l3evelopment 

The Contractor Evaluation Program was analyzed to develop a functional requirements 
document 



Business Case Analysis 

The business case analysis focused on an assessment of the alternate approaches for 
implementing Past Performance policy. Information was addressed in three areas: 

• The current processes and systems that deal with contractor past performance; 

• The recent changes in the processes and systems that are directed by the FAR 
and the proposed changes to the DFARS; and 

• A proposed approach for dealing with contractor past performance issues, 
referred to as the Contractor Evaluation Program 

The analysis covers process mapping, automated data information system analysis, and a 
comparison of the differences between the proposed Contractor Evaluation Program 
model and the FAR/DFARS approach. 
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SECTION ill. A. Government Perspective 



1. Establish Past Performance information Definition 

Past perfonnancc information is relevant infonnation regarding a contractor's actions 
under previously awarded contracts. It includes: 

• The contractor's record of conforming to specifications and to standards of good 
workmanship 

• The contractor's record of containing and forecasting costs on any previously 
performed cost reimbursable contracts 

• The contractor's adherence to contract schedules, including the administrative 
aspects of performance 

• The contractor's history for reasonable and cooperative behavior and 
commitment to customer satisfaction 

• The contractor's business-like concem for the interest of the customer 

2. Develop Past Performance Information Structure 

The 35 systems and procedures we examined listed in section II contained or in some 
way dealt with past performance information but did not have comparable information 
system structures. To add to the challenge most of the databases contained the 
information in multiple categories e.g. company, contract performance, administrative. 
We therefore found it useful to devise a scheme to classify all past performance 
information within the context of all government information. This information 
structure is shown below: 
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Once the chain of infonnation leading to past performance information for use in 
contractor selection was established, we distinguished between past performance 
information collected, validated, and filed for use in future contractor selections, and 
past performance mformation that collected and used-possibly validated-during an oq: 
going selection. This distinction is depicted as follows: 



Conttnued from 



Past Performance 
Information for use 
in source selection 



Colieaea. vauoaied 
and filed tor use in 
future source 
selection decisions 



AW.V.V.VAV.V.! 
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selection decisions 
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(Of future use 



InluditBDon 
from direct 
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Intormation 
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3. Identify Existing Systems and Processes 

Criteria derived from OFPP PoUcy Letter No. 92-5, FAR (FAC 90-26), OFF? "A Guide 
to Best Practices for Past Performance," and interviews with govemment and contractor 
users, system managere, and process owners were used to specifically identify existmg 
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govenunent past performance information systems, as opposed to systems that just had 
past performance information elements. 

Building on the definition of past performance information, the following criteria was 
used to specifically identify existing past performance information systems: 

• Inforaiation is collected, validated, filed, and dissemenated for the specific 
purpose of supporting fiituie contractor selection decisions; 

• Opportunity is provided for review, conunent iand rebuttal of the information by 
the contractor, 

• Provisions arc established to resolve disputes between the contractor and the 
government; 

• Information is subjected to the same controls and safeguards as other 
information used in contractor selection decisions; and 

• System is in operation and cunently supporting contractor selection decisions. 

Thus, a past performance information system is an ongoing effort to collect and record 
past performance information for subsequent use in determining contractor eligibility 
and selection. 

Using this definition, we identified three types of past performance information systems- 
- which include six distinct systems from the list of 35 systems we reviewed. The three 
types of past performance information systems are: 

• Performance appraisal systems which contain contractor evalutions prepared by 
cognizant government officials; 

• Performance tracking systems which draw on quality and delivery data from 
existing databases, and 

• Performance certification systems which establish specific criteria which are 
applied for purposes of identifying high levels of perfonhance exhibited by 
certain contractors. 

Using this structure, the following past performance information systems were 
identified: 

• Performance appraisal systems 

• Contractor Performance Assessment Reporting System (CPARS) developed 
and used within the Air Force 

• A&E Contract Administration Support System (ACASS) developed and used 
by the Army Corps of Engineers 
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• Construction Contract Appraisal Support System (CCASS), also developed 
and used by the Army Corps of Engineers 

• Performance tracking systems 

• RedA^ ellow/Green (RYG) system developed by the Navy Supply Systems 
Command and used by certain procurement orginizations within the Navy 

• Automated Best Value Model (AB VM) devleoped and used within DLA 

• Contractor Profile System (CPS), also developed by DLA 

• Performance certification systems which include Blue Ribbon Programs which 
have been implemented by some procurement organizations 
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Performance 




Performance 
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Performance ^praisal systems, such as CPARS and ACASS, generally cover a wide 
range of evaluation factors-CPARS addresses 14; ACASS rates 11. Performance 
tracking systems generally focus on two or three factors. The difference in a number of 
factors is generally due to two reasons: tracking systems are associated with higher 
volume, relatively small dollar acquisitions (S25 - $500K), and evaluations may be 
conducted on less than the whole contract requirements. The performance appraisal 
systems differ from tracking systems in both respects in that the evaluation is for the 
whole contract and it is used on a relatively low-level of high-dollar transactions. 
CPARs uses interim reports so in one sense is also based on less than the whole 
contract, but the final CPARS report card for a contractor is accomplished for the whole 
contract requirement. The existing systems focus on a specific segment of the DOD 
acquisition program in terms of contractor dollar value and product/service. 
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Performance certification systems are generally for the same level of complexity and 
dollar value as tracking systems. They build on the data in tracking systems but go a 
step beyond rating and ranking contractors. Performance certification systems actually 
offer pie-established evaluation standing in the contractor selection process. Certifying 
contractor performance requires a broader/deeper level of information than is obtained 
in tracking systems. 

4. Coverage of DOD Acquisition Program by Existing Government Past 
Performance Information Systems 

By 1997 the proposed DFARS policy provides that past performance information must 
be collected for all contracts over $100,000. There is currently a large number of active 
DOD contracts and it is important at this point to discuss the coverage that existing past 
performance information systems provide relative to the total DOD acquisition program. 
It is this collection requirement that is the biggest challenge DOD is facing in 
implementing past performance policy. 

This table shows the limited coverage that is available from current systems from dollar 
value and product/service perspective. 



Contract Dollar 
Values 


Research & 
Development 


Services and 
Construction 


Supplies and 
Equipment 


>$5M 






CPARS* 


>$100K 




ACASS 
CCASS 




>$100K 






RYG* 
ABVM^ 
BRCP 



^ CPARS is also used on major R&D programs 

^ The contractor pcrfonnancc data only from RYG can be used for transactions above $ lOOK 

^ AB VM can be used for higher dollar transactions where the higher dollar value is due to greater quantities 



Gaps in coverage are evident. Supplies and Equipment have the most applications in 
terms of past performance evaluation systems. 
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5. Systems Assessments and Key Characteristics Evaluation 

The system assessments and key characteristics evaluations required by the Statement of 
Work as to the success of the existing government past performance information 
systems follow. 

System Assessment-Red/Yellow/Green (RYG) This is a Navy system designed to help 
reduce the risk of receiving non conforming products and late shipments. RYG 
classifies the degree of risk by assigning a color code to a contractor's historical quality 
and delivery performance in individual Federal Supply Qassifications (FSCs). Red is 
high risk, yellow is moderate risk, and green is low risk. The system provides 
procedures and an automated system for incorporating these classifications into 
contractor selection decisions. 



In addition to the color mdicators, the system provides price adjustment factors that 
reflects the additional cost to the government for actions needed to reduce the risk of 
receiving non conforming products and late deliveries. When added to a red or yellow 
offeror's price, the adjustment factor may displace the low offeror in favor of an offeror 
with a better product quality and delivery history. The price adjustment feature of the 
system generally has relevance to the smaller contracts because the adjustment factors 
represent a fixed amount and this amount tends to lose significance as the contract value 
approaches $100,000. 



The RYG system tracks the quality of items delivered under specific line items and does 
so by relating discrepancy information observed and reported by government personnel 
at sites that accept and stock the material. This information is used to calculate a rating 
for the contractor's specific plant location and for the specific FSC. An opportunity is 
provided to each contractor to review their ratings, and to challenge the basis upon 
which the ratings were determined. Any differences are resolved between the 
govemment and the contractor. 



The ratings and the related price adjustment factors are then available for use by 
govemment officials as a factor in contractor award decision, provided that the 
solicitation informed the contractors that this past performance information would be 
used for making the award. The system does not currently cover delivery information, 
although activity is underway to include this information. And the system does not have 
the capsbHity to track the in-service quality and reliability of items after delivery and 
acceptance, although this type of information can be retrieved from the database upon 
which the RYG system draws its data. 
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The RYG system has been in operational use for over five years and implementation and 
enhancements are continuing. The following was determined during the course of the 
study: 

• The RYG system was available at 17 of the planned 41 sites, but some are 
closing 

• Plans for expanded use at more sites are unclear and unscheduled 

• Use to this point has largely been by advertising the system's value versus 
directing its use 

The RYG system includes quality data and is currently adding delivery performance, 
which is not yet operational. It combines the data with an algorithm that produces a 
color indicator (red, yellow, or green) and a Technical Evaluation Adjustment (TEA), 
which is a price adjustment added to bid price of contractors with a yellow or red rating. 
As the dollar value of the contract increases, the effect of the TEA in an award decision 
decreases. For example, contract awards over $100K do not use the TEA feature. 
However, color code ratings can provide a past performance indicator for any contract 
value. When delivery performance is incorporated in the system, two sets of past 
performance indicators will be provided for each FSC in which a contractor does 
business - one for quality and one for delivery. RYG Data is downloaded monthly to 
tt)(e using acquisition offices. Contractor have electronic access to and can read their 
ratings. RYG gives indications that design objectives are being met, but it is too soon to 
judge ultimate success throughout the Navy with any certainty. 

System Assessment - ABVM: This DLA system is also designed to cover specific 
equipment and supplies with FSC's and firm specifications. In this respect, coverage of 
the system is similar to the RYG system. It includes information on the reported quality 
and on-time deliver of specific contract line items and uses this information to calculate 
a score for each contractor's site and for each FSC. An opportxmity is provided to each 
contractor to review their scores, and to challenge the basis upon which they were 
determined. The scores are then made available to buyers for use in making award 
decisions. The ABVM also has the capability to include the results of random testing 
for items accepted and maintained in stock. 

The ABVM is a module in the DLA Pre-award Contracting Systerai, which is the 
migration system being used in the development of the Standard Procurement System 
(SPS). Initial implementation of ABVM started in 1995 and is currently underway at 
DLA sites. Consequently, it was not practicable to conduct an objective assessment of 
the system from the perspective of users. A user survey is currently being plaimed by 
DLA to assess the performance of the system and to solicit ideas for system 
enhancements. 
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The ABVM system is in the early stages of operational use. For example: 

• Defense General Supply Center started ABVM operation in July 1995: 

• Training programs arc currently underway 

• Other DLA Centers will have installed the system in 1996. 

The ABVM system includes quality and delivery performance data which are combined 
with an algorithm to produce a score for each contractor in each FSC. Past performance 
scores are used as a tool in making a comparative assessment of price and performance 
risk. ABVM information is provided to buyers through DPACS. Contractors can read, 
rating through in EBB . ABVM replaces the Quality Vendor Program (QVP) as the 
principal system used by DLA. QVP is a performance certification type system rather 
than a performance measurement system. DLA shifted to the ABVM because QVP was: 

• Too burdensome to administer; 

• Covered only a small portion of the supplier base; 

• Ended up with two ratings-certified and non-certified rather than a more 
comprehensive ranking of the suppliers 

QVP is still authorized for use for specific FSCs or selected service requirements by 
individual contracting offices. ABVM gives indications that design objectives are being 
met. but it is too soon to judge ultimate success throughout the DLA with any certainty. 

System Assessment - CPARS: This is an Air Force system that applies to major 
acquisition programs above $5 million. It involves a periodic assessment by 
government officials responsible for the overall program and covers factors that include: 

• Product / system performance, including system engineering and software 
development 

• Schedule 

• Cost performance 

• Product assurance 

• Test and evaluation 

• ILS program 

• Management responsiveness 

• Subcontract management 
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Each report includes a description of the program, a statement describing the 
contractor's effort, a narrative that addresses the performance of the contractor during 
the period, and a rating for each of the evaluation areas listed above using four color 
codes-red, yellow, green and blue. Performance appraisals are provided to contractors 
for their review and conmient, and then reviewed by the Government evaluator who 
may adjust the initial appraisal. 

At the present time CPARS is a manual system. Completed reports are identified as 
"Source Selection Information" and filed in libraries maintained at AFMC organizations 
that initiate the assessment report. Information is retrieved for use in source selection 
decisions by contacting the cognizant CPARS focal point 

Some initial action has been taken to automate the CPARS process utilizing Lotus Notes 
- as well as to extend coverage to small systems, services, science and technology, and 
operational contracting. 

This Air Force system was designed for major system acquisitions with a low, volume of 
transactions, extensive performance measurement categories, and is a manual system 
kept in files at Product Centers. It provides strong support to the source selection 
process by conununicating contractor strengths and weaknesses; it covers relevant areas 
of performance; uses contractor data; and is updated every 12 months. It may also 
provide out-of-cycle reports. It provides relative performance feedback to contractors 
across all measurements. CPARS very consistently performs its intended purpose as 
reports are based on first-hand data controlled by program offices. 

System Assessment - ACASS & CCASS: ACASS is a system used by the Army Corps of 
Engineers which covers architect and engineering services related to construction (Code 
C in the coding structure used by the Federal Procurement Data System (FPDS)). 
Evaluations are prepared by professionals who review and accept the work. Principal 
evaluation areas include: 

• Thoroughness of site investigations 

• Quality control procedures and execution 

• Accuracy of plans and specification 

• Clarity and completeness of the plans 

• Overall management and adherence to schedule 

• Compliance with cost limitations 

• Suitability of design or study results 

• Environmental suitability of proposed solution 
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• Cooperativeness and responsiveness of contractor 

• Quality of briefings and presentations 

Evaluations are prepared at the completion of contract efforts and ratings arc assigned in 
three categories - outstanding, satisfactory, and unsatisfactory. Contractors have the 
opportunity to review and challenge the evaluations. And completed assessments are 
maintained in a central database which can be accessed by officials who arc involved m 
contracting for A&E services. 

CCASS is also a system used by the Army Corps of Engineers which covers the 
construction of structurcs and facilities (Code Y in the coding structure used by the 
FPDS). Evaluations are also prepared by professionals who review and accept the work. 
Principal evaluation areas include: 

• Quality of work (includmg eleven sub-factors) 

• Timeliness (mcluding seven sub-factors) 

• Effectiveness of management (including nine sub-factors) 

• Compliance with labor standards (including three sub-factors) 
•» Compliance with safety standards (including three sub-factors) 

Evaluations are prepared at the completion of contract efforts and ratings are assigned in 
five categories-outstanding, above average, satisfactory, marginal and unsatisfactory. 
Contractors have the opportunity to review and challenge the evaluations. And 
completed assessments are maintained in a central database which can be accessed by 
officials who are involved in contracting construction work. 

These systems were originally designed to facilitate selection of "qualified" A&E and 
construction contractors. The systems were recently expanded to provide for 
incorporation of past performance information into contract award decisions. 
Evaluations are performed by government professionals responsible for reviewing and 

accepting work, namely: 

• Administrative Contracting Officers 

• Contracting Officer's Representative 

• Other Receiving Officials 

• Resident Engineers 
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Reports are reviewed with contractors and entered into a central database via computer 
or mail. Access to the data is provided to COE elements; contractors do not have read 
access to the rating information. 

The CPARS approach used for major acquisitions has been tailored to be suitable for 
small systems, services, and R&T. The tailoring is primarily in the evaluation factors 
that are addressed in each case. Some initial work has also been done in automating the 
records that would facilitate the conmiunication with contractors and the filing and 
retrieval of information. 

System Assessment-Contractor Profile System (CPS): The Contractor Profile System, a 
DLA system that is currently available for use, did not fully meet the other criteria for 
past performance information systems. Work is underway by DCMC to enhance 
MCX:AS data extraction. DCMC's Contractor Information Service which is currently 
underdevelopment will encompass CPS and its system enhancements which are in 
progress. The CIS is discussed later in this report under planned past performance 
information systems. 

Key Characteristics Evaluation: The results of comparing the five existing DOD Past 
Performance Information Systems to the 16 key characteristics provided in the 
Statement of Work is shown below. 
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Existing Past Performance Information Systems 


1 System Evaluation 
1 Factors 


RYG 
Navy 


ABVM 
DLA 


CPARS 
Air Force 


ACASS 
CCASS 


CPS 
DLA 


Qsta tyitam dMign • eMmlnd 

or nofi-eentrallad 


Cemralized 


Non-centralized 


Centralized 


CenlrallzBd 


Centrafized 


Kinds of data usad • 
Govammont, privsta 


Quality and 
delivery 


Quality and 
deiivery 


Cost, schedule. 
tectL perform. 


Cost, schedule, 
tecti. perform. 


Access to exist * 


IntagrKy of data Mantlly al 
. aoureaa 


0mm from ex* 
isting databases 


Drawn from ox* 
isting databases 


EvaL by respon • 
sible officials 


EvaL by respon - 
8ft}leofndals 


MOCAS.PASS. 
andOPACS 


Acetiraqr 


Ensured by con - 
tractor review 


Ensured t>y con * 
tractor review 


Ensured by con • 
tractor review 


Ensured by con * 
tractor review 


Limited to accu - 
racy of data input 


Cunanqr - fraquaney of update 


Montttfy 


Monthly 


Annually & con * 
tract completion 


Atcontnct 
completion 


Based on source 


RofnodUil pracaaa by oontradofa 


Yes 


Yes 


Yes 


Yes 


No routine 
process in ptaoe 


Avaitablllty of Infonnatlon far 
source selection 


Via computer 
tenminal 


Via computer 

tenminal 


From CPARS 
focal point 


Via computer 

termiruJ 


Via computer 
terminal 


Confidantiallty 


Yea 


Yes 


Yas 


Yes 


Yes 


Subcontractor tnvohfament 


No 


No 


Yes 


No 


No 


Maintaining Identity of eon 
tractors ttiat are aequlnd 


Yes 


Yas 


Yas 


No 


No 


Falmssa 


Yes 


Yes 


Yes 


Yes 


Fairness 


Due process 


Yes 


Yas 


Yea 


Yes 


Due process 


Lack of past perfocmanoa 


Neutral rating 
used 


Average score 
used 


NA 


Notindudedm 
database 


roi aooresseo 


Threshold of applleablltty 


Primartty below 
$100K 


Primariiy below 
S100K 


SSMand 
atxjve 


Over$2SK 


DLA contracts & 


Capability of attribution 


Pnotected 


Protected 


Protected 


Protected 


No pedonnanoe 
evaluation info. 


Penalty 


Info, used In 
source selection 


Info, used in 
source selection 


Info, usad m 
source selection 


Info, used in 
source selection 


Info, used In 



Detailed evaluations for each system are included in the appendix section. 



6. Systems Under Development 

Each of the Services and DLA have initiatives underway to implement DFARS that are 
aimed at expanding the past performance information available for use in contractor 
selection decisions. 

The Air Force is examining an automated version of CPARS. The Navy is exploring a 
Contractor Evaluation System. The Army is developing a Performance Information 
Management System. DLA is developing the Contractor Information Service. 

The Contractor Information Service design goal is to make DCMC's knowledge and 
experience more accessible to its customers, including: 

• Near term - enhance MOCAS data extraction capability and develop past 
performance input screens 



• Mid term - merge three existing systems (CPS, PASS, DSIS) into a single 
information system over the next two to three years 

Information will be organized on a company-wide basis with a capability to "drill 
down" to divisions and plant facilities. Coverage envisioned at this time includes: 

• Principal product lines and unique production capabilities 

• Company organization and key personnel 

• Sales, earning and financial health 

• Past performance history - trends, data, and commentary 

• Pricing information - rates and factors 

• Systems and processes status - risk assesments and corrective actions 

• Prior reviews - GAO, IG, DCMC, DCAA, buying activities 

• Acquisition strategy "lessons learned" 

7. Experience of Other Government Agencies 

A total of 15 other non-DOD Federal Agencies wexe reviewed m terms of their approach 
to past performance information systems. The diversity of agencies provided extensive 
coverage in terms of the nature of products and services contracted for as well as 
missions performed. 

Our observations are based on contacts with officials in several agencies and on a 
review of documentation on past performance implementation. Most agencies are 
implementing past performance by passmg the OFPP Guide along with minimal 
guidance. Some exceptions to this include: 

• Energy-which issued a 10-page Acquisition Letter to accompany the OFPP 
Guide 

• GSA-which issued an Acquisition Letter in March 1993 (which is being 
updated) plus Federal Supply Service specific guidance m a separate Acquisition 
Letter issued in October 1995 

• Transportation-which puts all guidance in the "Transportation Acquisition 
Manual" 

Generally, the evaluation form in the OFPP Guide is provided for guidance purposes. 
No guidance is provided on how to tailor the evaluation to size, content, and complexity 
of the contractual requirements. Little additional guidance on known problem areas is 
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available (e.g., meaning of neutral rating for contractors with no past performance). No 
automated databases exist, but some agencies are planning to investigate. Specific 
concerns include: 

• Protests and ruling by appeals boards and courts 

• Number of open, active contracts that will need to be evaluated 

• Workload impact 

NASA will use past performance in source selection, but will not evaluate on-going 
contracts except as required for award fee determinations. NASA implemented a 
Contractor Performance Summary (CPS) in January 1992 which was based on the Air 
Force CPARS. CPS features included: 

• An evaluation on all award fee contracts above $25M 

• Evaluation of non-award fee contracts was discretionary (by the Centers) 

The CPS system was abandoned in March 1994 because the value added to the 
contracting process could not justify continuation of the system. In response to OFPP on 
the recent FAR changes, NASA will continue to use past performance as a standard 
evaluation factor in source selections (NASA has been doing this for at least 6 years), 
b^t will not create an Agency-wide system to require performance reports on active 
contracts. 

NASA elected not to require performance reports because: 

• Award fee evaluations capture approoximately 80% of NASA's procurement 
dollars (and these are exempt from the FAR) 

• Implementing an Agency-wide system would significantly burden the ^yorkforce 
without significant benefits 

At the $100K threshold, 80% of contracts would be non-award fee contracts and would 
account for about 20% of the procurement dollars. NASA estimates that a ten-fold 
increase in evaluations would be needed to evaluate the non-award fee contracts (from 
224 to 2404). OFPP requested that NASA reconsider their decision, but this is 
apparently unlikely. 

The Federal Supply Service in GSA has issued policy on use of past performance 
information and on a system for routinely recording this information. Guidance was 
provided by the FSS Acquisition Letter FC-95-7 of October 19, 1995, "Use of Past 
Performance as an Award Evaluation Factor - Routine Stock and Special Order 
Programs.'' The guidelines allow a contracting officer to efficiently use the quality of an 
offeror's past performance as a factor in a contract award decision. Past performance is 
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to be considered along with price and applies to negotiated acquisitions in excess of $1 
million. 



The process provides for. 

• Supplier Rating Reports used to evaluate contractor performance by the 
cognizant Office of Quality and Contract. 

• The Administrative Contracting Officer opinion, supported by a sununary of the 
Supplier Rating Report, is provided to a Procuring Contractoring Officer upon 
request 

• The ultimate award decision must be based on the Contracting Officer^s 
judgement and not just the results of a mathematical calculation. 

The existence of well-developed policy was very limited. Taken as a whole, these 15 
agencies* systems, with the exception of tiie GSA Federal Supply Service, appear to 
represent less structured and more ad hoc past performance evaluation programs. 
While represented in this report at a sunmiary level, most of these agencies* programs 
appear to represent immature, unstructured approaches to evaluation. 
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SECTION Hi. B. Industry Perspective 



This section presents the results of industry benchmarking and our review of industry 
supplier evaluation programs. 

Benchmarking 

The benchmarking phase of the study was accomplished by conducting on-site visits 
and by reviewing information in the Arthur D. Little Supply Chain Management 
practice database. 

The names of companies we visited during the course of the study and the industries 
they represent are listed below. Notice that while manufacturing is heavily represented, 
industries dealing with electronics, process industries and companies performing 
logistics-like activities were included in our research to provide both breadth and depth 
in terms of industry types. 



1 Companies Included in the Supplier Evaluation Database 




Alien-8mdley 


Process Controls Equipment Manufacturing 


Baxter 


Pharmaceutical Manufacturing & Distribution 


Black and Decker 


Consumer Goods Manufacturing 


Boeing Defense & Space Group 


Aerospace/Defense Manufacturing 


British rail 


Transportation 


Fisher Scientific 


Industrial Distribution 


Ford Motor Co. 


Automotive Manufacturing 


McCorrrack & Co. 


Consumer Goods Manufacturing 


McDonnell Douglas 


Aerospace/Defense Manufacturing 


Mobil Corporation 


Process Manufacturing 


NationaJ Semiconductor 


Electronics Manufacturing 


Rockwell Defense Eiectronks 


Aerospace/Defense Manufacturing 


W.W. Grainger 


Industrial Distribution 


U.S. Postal Sen/ice 


Transportation 



Companies researched for benchmarking purposes included those producing consumer 
items as well as those in the defense contracting community. In addition, companies 



manufacturing component parts were also included since supplier certification is often 
performed down to the part level in a system. 



A key finding of our industry research is that in best-of-class supplier evaluation 
programs there is a distinct supplier approval process keyed to associated risks. Also, 
the supplier approval process outcome results in a consolidation of suppliers, which is a 
necessary condition before a business relationship can take place. 

Industry Supplier Evaluation Programs 

The information in this section is organized according to the key features we found in 
industry supplier evaluation programs. The nine key features, which were identified in 
companies that are recognized as "best of class" among supplier evaluation programs 
along with their purpose, scope, and selected implementation features are summarized in 
the following table: 



\ Evaluation Program 
1 Component 


Purpose 


Scope 


Selected hnplementation Features 


'^^^^^^^^^^^^^ 


1 


• Algn««|sijfb««mtgyto 
eorpanM MMoif 

• Mww^n ttw<uppiybii<tOMhli»» 

• M«Mo«ttwwppfyb«Mloa«it« 
\m¥9f^}m acM>Mi iMHt loial ootf. 
ftin oonipslliwi ■dwttQS 

• Iwpww coKinuoMriy 


Compan^^^Mda 


• S(CToC supply ba—.owawl and 
vrfthfet oommodty sagmams 

• Commodttaa Manaoamant SMagy 




• toantlficction of kay auppfiam 

• LonQ^ann partnanhip ttraiagiaa 

• ISO 9000. Baidrige. other oartMlcBtlQn 
faQMratnantt 




• EvMkMiv ■i4iplltf pcfloRimw on an 
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Supply Based Management Process/Supply-Based Strategy 

Notice that the scope of application ranges from all suppliers to critical suppliers. 
In terms of **best-of-class*' benchmarldng findings, the following three principles, 
derived from the first key feature listed above, stand out 

• World class supply chain orientation 

• Supply base improvement strategy 

• Explicit supply base management process 

Companies that adopt these principle have seen a significant increase in operational and 
organizational performance. The emphasis on a world-class orientation moves 
organizations from a prescriptive "meet the spec" environment to a ftilly collaborative 
internal and external team environment which emphasizes process rather than 
specifications. The emphasis on developing an explicit supply base improvement 
strategic and management process raises the level of supplier performance, reduces 
supply chain costs, and moves staff and supplies into new roles which change over time 
from a largely reactive to a proactive orientation that reinforces continued 
improvements. Conspicuous in this new perspective is a systems approach to delivery 
of world-class products and services. 

Supplier Performance Measurement 

Ongoing supplier performance measurement is a central feature of supplier evaluation 
programs. Supplier perfomiance measurement is generally performed for a small 
number of critical data elements, such as quality, service, delivery, and cost Each 
business unit defines what constitutes product quality, service, and delivery 
performance, as well as the appropriate measures for each of these. 

Prior to beginning supplier performance measurements, the customer's performance 
standards and requirements are communicated to suppliers. These include how often a 
supplier will be rated and how ratings will be used and conununicated. 

Detailed profiles of ratings are generally available on-line. A supplier's data is never 
shared with anotiier supplier. Two examples of these profiles follow: 
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In this profile, a supplier's performance index (SPI) and an average for the commodity 
group arc calculated Each commodity group has a cutoff or "rcdline" Suppliers with 
an SPI above the redline arc not eligible for awards. • 



A sample supplier performance report used by a national distributor has four categories, 
three of which pertain exclusively to tfie supplier's performance in terms of quality, cost 
and delivery (timeliness or schedule). 
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The last, sales, pertains to the buying company's performance associated with a product 
line involving a particular supplier. Measures in the "criteria" column of the report are 
accompanied by data in a format which has the capability to indicate trends-if any 
exists. In addition to using measures applicable to operating units, the report also 
includes data in dollars-suitable for use by upper management. 
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Suppliers Performance Measurement Feedback 

Another key feature of a supplier evaluation program is focused on feedback processes 
and improvements in communication. Feedback to suppliers is a very important 
ingredient in an effective supplier evaluation program. This is mutually beneficial as it 
provides needed information on quality to suppliers for their own improvement 
processes. Best-in-class companies provide feedback to their suppliers on their 
performance results for tiie purposes of improving future performance. While feedback 
is the key to improvement, an effective supplier evaluation program will have to contend 
with both the nature of specific feedback as well as the frequency. Many organizations 
utilize a formal "report card*' process to provide suppliers feedback in a structured 
fashion periodically. .Many companies meet in person with their suppliers at least once 
a year to inform them of their evaluation results, identify areas of improvement, and in 
more advanced situations, develop an action plan for improvement. Companies also 
notify their suppliers more frequenfly by on-line services, telephone, or letter about their 
performance. This feedback is critical since it gives both parties the opportunity to 
improve the product, reduce costs, and improve service 



Supplier Quality System Assessment 

The foundation for a supplier evaluation program lies in an active, thorough, on-site 
evaluation of a supplier's approach to the installation and use of an effective Quality 
SystenL Supplier quality systems assessments are often based on rigorous standards 
such as the ISO 9000 series of standards. A key feature of the ISO series is registration 
of a company or production element virith a third party organization which monitor 
compliance to tiie registered standard. Purchasers of products and services from ISO 
registered companies are assured that the registered company has a documented quality 
system in place. Some ^preaches to assessment are developed in-house using ISO 
9000 (or other applicable standards for the industry) or the Malcolm Baldrige National 
Quality Award criteria. However, the most objective ^preaches at tiiis time rely on 
third-party certification including on-site evaluation, subsequent registration, and 
periodic re-evaluation. 



Part-level Certification 

Use of this type of approach requires accurate historical data on supplier past 
performance. An important outcome, often not explicitiy stated, is the change m the 
relationship which occurs as a result of becoming a certified supplier. Generally, 
companies requiring supplier certification often experience a decrease in the number of 
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qualified suppliers. The remaining suppliers, then, have an opportunity for a more 
stable business relationship. 

Supplier certification tends to bring increased benefits for both the certified supplier and 
the customer. For the supplier, it can mean additional business, single or lead source 
within a commodity area. For the customer, it can mean significant cost savings as a 
result of being able to use parts received fi-om certified suppliers because certification 
can eliminate costly incoming inspection and associated costs. 

The best-in-class supplier evaluation programs usually certify to the item/part or family 
of parts level. Most companies have the goal of certifying all of their key parts and 
products. However, they typically start with a manageable number of critical parts and 
then expand the program to include all of the critical items as well as those that have the 
potential to reduce operating costs. Some companies interviewed during the course of 
the study had certified virtually all of their products or were on their way to certifying 
all critical parts. 



Total Cost Assessment 

An emerging trend in the supplier evaluation arena is the use of a 'total cost 
assessment" approach which attempts to c^turc all of the acquisition and consumption 
costs associated with doing business with a particular supplier. Acquisition costs arc the 
costs of a supplier's activities to process and deliver an order supplier's material-plus 
profit Consumption costs are the costs of the customer activities-labor and overhead— 
to process a supplier's shipment through the customer's system. Effective total costs 
assessment processes usually rely on activity-based costing principles. 

Activity Based Costing techniques are used to acquire the best value by estimating the 
total costs of doing business with different suppliers. The ^true" lowest bidder is 
sought-and bids account for all costs including quality, cost, and delivery. Customers 
identify historical non-productive costs resulting ft"om supplier non-compliance with 
customer's mode of operations. Some supplier non-productive events that are "charged" 
to the supplier are: 
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Quality Events 



Source rejection 

Inspection resubmittal 

Return to supplier 

Material review 

Shop floor rejection (latent 
defect) 

Corrective action request letter 
Supplier stop notice 
CECA action 



Scheduled Events 



Eariy delivery 
Overshipment 
Late receipt 



One important use of a total cost assessment is the adjustment of bid prices from 
suppliers using a Supplier Performance Index (SPI). The index is developed from a 
ration that estimates the true cost of supplier bids. An example appUcation of the 
Supplier Performance Index concept is illustrated below. 
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Supplier Evaluation and Development: 

There are two primary purposes to supplier evaluation and development initiatives: 

• Communicate supplier performance standards and requirements; and, 

• Educate suppliers on the supplier improvement process. 
• 

The scope generally covers all suppliers, but especially critical suppliers. Companies 
typically communicate their guidelines and standards through published documents and 



41 



formal supplier education programs. This is a highly proactive process in which 
companies view their suppliers "as their customers.** 

Supplier Approval 

A robust supplier approval process incorporates multiple data sources, focuses on 
quality, is documented, and is shared witii suppliers. Supplier mformation gatiiered 
during the evaluation may include general business standing, service levels, 
distribution/logistic capabilities, supplier specifications/product brochures, company 
networicing, and existing like-product data. 



An example of the scope and depth of supplier approval programs is presented in this 
chart: 
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Note: Example from Manufacturer/distributor operating under Current Good Manufacturing Practices (CGMP). 
Variations in who has regulatory responsibility, provides specifications, labeling, design, and change control 
responsibility, tradcmaik, etc. 



Approval is formally documented to cover approved locations; any required reports or 
data; a list of processes approved; additional relevant quality information; and sign-off 
by business area teams. 
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Suppli^ Recqgnltion l^l^iP!^ 

Many commercial firms acknowledge supplier performance with some type of 
recognition program. How this recognition is achieved varies, but an important outcome 
is the strengthening of customer-supplier relationships. Many companies present their 
best performing suppliers with an award, while others less formally send thank you 
letters. Most suppliers strive fcB^suc^ as more 

business fitem the customer giying thc^^^ 
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Section III. C. Contractor Evaluation Program 



1. Introduction 



developmeot of ««»xpts to explore m a Bno^^^ 

facilitate the workshops. The third phase invoivca ^wicuum |^«!^„„^-„ts and 

worShops and coUecting perspectives and insights concerning To-be concepts and 

Evaluation Program. ' 
n^no^owever the workshops, with ARSSG representatives, added value to the 
ZZ^PZ^ (APCLogistics. Economic S^^'^'^y^'^^^T^ 

we were able to test and explore the impUcations of the vision of a 
^uZZcs for the DOD acquisition management and user commumties. Tlie workshop 
fSso pr^d^a f^rn for identification of mUestones supportmg the vision and 
discussion of actions to be taken. 

•Die workshop approach provided an opportunity to present the status of current activity 
S^^orJ^f ^vemment L industry arenas and led to the ^^-^"^JSuowing- 
impolit ff ctors for the CongactorEval^^ deluding the foUowmg. 

. A key objective for industry in adopting supplier evaluation programs is to 

increase competitiveness. Characteristics associated witii mcreased 

competitiveness include: 

• Reduced costs 

• Reduced cycle and response times 

• Improved operational efficiencies 

• Increased customer satisfaction and loyalty 
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. Reduced inventory (improved inventory turnover) 

• Increased revenues 
Tteekeyfacton^areinherentinthebuyer-suppUerreladoi^j^^ 
colhboration, competency, and conHnuous improvement. Characteristics 
associated with these include: 

• Willingness to invest resources 

• Quality products 

• Service 

• Responsiveness 

• Technology 

• Corporate culture 

• Business area and management strategy 

• Qualification Assessment (single quality system) 

• Performance measurement 

• Performance feedback 

• Item certification 

• Total cost assessment 

• Supplier recognition 

• Supplier evaluation and development 

• Supplier approval 

. The purposes for past performance in DOD: 

. Evaluate risk of performance by providing information which can l« 
uLdJn Lking ttade-off decisions for what is the best value m the sour« 
LTectTon^esrTTus information can be used in the aw^d of the miUal 
ronSeT^Sofoptions^andtheissuanceoftaskanddehveryorders. 

. Develop ac,uisiHon strategy by helping in the <;-i^^"^^^^^^^^ 

type and source selection factors, e.g., the mid-1980s overuse ot 

development contracts led to many cost overruns. 
• Afonag^conrrflcrorpe/fonnanc. by providing information to^ 

variances form estabUshed tolerances in the existing program 
. Improve contractor performance by providing feedback to the contractor 

Ibout plrfonnance whfch wUl allow the contractor an opportunity to 

improve its performance. 
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• Allocate oversight and review resources by identifying those contracts or 
aspects of contracts in which experience dictates there have been 
problems, and employing our oversight resources in those areas. 

2. Contractor Evaluation Program 

The breadth, depth, and complexity of requirements is a major challenge to those 
involved in DOD acquisition programs and to those proposing solutions to issues-such 
as the past performance policy implementation issues being considered in this study. 
The Contractor Evaluation Program we designed is aimed at simplifying the past 
performance implementation effort facing the DOD as well as to improve the 
effectiveness of this effort. The program is conducted by cross-fimctional Business 
Area Teams that start locally and may extend across organizations and services, as 
appropriate. 



Overview 

Implementation of the recent poUcy on contractor past performance requires a 
recognition of the business environment and existing acquisition systems, to include: 

• The total size of the defense business 

• The wide range of products and services for which contracts are issued 

• The large number of procurement orgamzations that issue contracts 

• The existing procurement process 

In the aggregate tiiese factors define a conglomerate that is engaged in an extensive 
number of business areas. In recognition of these factors, and witii die overaU goal of 
reducmg the cost of doing business, the Contractor Evaluation Program is designed to: 

• Develop a Business Area Plan, including defining common business areas 

• Develop a Business Area Strategy that makes sense for tiie particular business 
area 

• Develop a Business Area Evaluation Process to unplement contractor evaluation 
in the context of the business area 
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l^dXs the products and services acquired by all the services and agencies. 



Business Area Plan 

There are four aspects to the business area plan 



Define your business area . 

ronduct internal, i"Hn5;trv. and contractor analyses 



Develop a business area reso urce center 



Form business area alliances. 



illustrate the process for defining your business area follows. 



purchased in DOD is the basis for starting this 



The wide range of products and services 
,van,pi^ nf definin g a busine ss area. 

These product and sen/lces nrs bought In DoD'. business areas. 




48 



Theseareso.eofthe.aJorac,uisiaonorg^^^^^^ 
thebusiness area definition P..c^^*^^^ 

categorized by n rfrm rr-*"'^ hnt^/post/camp. anaA ^ 
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— General Supply _ 
Center. Richmond 

—Consmictton Supply 
Center. CoUntxis 

— ttectrorics Suoo*v 
Center. Dayton 

— Fuel Supp»v Center, 

Aiexandria. VA 
— mdustrtal Supp^/ 

Center, PWla. 
^ersomol Support 

Center. PNla. 
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Pacific. Alameda 
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Office. Alexandria 
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organizations. 

Contractor Evaluation Program 
Business Areas 
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A business area depicted at a conunodity level: 




uaia and Lubricants 



I 



the business area teams suppUed with ^^^^^ have an on-line or 

their business area analysis. 

STt, facUifc, profitabUity. and sue/grow* mformanon. 
Producers Prices, and Prices Index. 

Issues typically addressed include: 
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What is the ^ experience and future tequire,n«..s of the government in this 



^ an= the relevant characB»tics of the industry in tenns of size, growth, and 

competitive forces? . . ■ j 

. whatisthecurrentpositionofthekeycon.ractorsmthen.dustry. 

Son- of the toors .hat are typically involved in this analysis include-. 
(1) Internal analysis 

• Current contractor/supplier base 

• Government's past experience 

• Expenditures over time 

• Internal acquisition costs 

• Projected requirements 

(2) External analysis (industry) 

• Market size and growtii 

• Capacity and utilization 

• Market share of principal contractors 

• Industry profitability 

• Cost structure and drivers 

(3) External analysis (contractor) 

• Customer base 

• Position in the mdustry 

• Commitment to industry 

• Quality and service performance 

Fo™ tu^ness ares ^llances: Regarding the b-m«s ajea as ^^y^^^^j:^ZL. 
denies Asynergy that exists within^ ^^^^^=^0^ he energy in the 

At its fuUest expression, a business arta »^ f =^'=3 inefficiency 

^iTfSSreteT^"-^^ 

contractor base. 
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,occ th*. Services and DLA are formed to coordinate their 
ifbusiness area tean^ aero s^e^^^^^^ 
business processes and present oneiace rouicu 



Contractor Business Area Strategy: 

selection decisions. 



Tnte prating the Business Area 



Business Area Goals 



Uses for Past Performance Information^ 



commodity group, medical services. 



Horizontal integ 




Through uni<,u. horizontal '-XZ^^Z'^t^^oZ't^^ 

help DOD organizadons how to meet f f '"^^.^^j,,^ J, ^^egy is 

past performance to a broader business-like viewpomt. 
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B.^„A«.Go...F™.o„ranalysis.weide„af.edgoals«^a„effecav.s«.«gy 

might include: 

• Develop a world-class orientation 

. Maintain total quality with focus on continuous improvements 

. Increase the number of high-quality suppUers 

. improvecontributionstocorporateprofitabiUty/operations 

. imUentateamapproachinten^llyandextern^^^^^^^^ 

integral part of the team 
• Accreditation of key. critical suppUets .Hriral 

uMprosmn^°--i«'0^f^^^^^^^^^?,occs.: measurement 
too one or aU of the ttoee ate^ "S^^^^^^^ deUbe^tions may be a«t 
certification, and '^^T^^^^l:'^ ,rJc^s^^ 

products/services. 

f T.i,., .nildtation/awa^ """""ch to selecunj-conwactog. 
p^tabllshed (optio" exercised and IDIA deasionsx 



nraanizalion's p erformance goals 



I Recognize sun wior pertomiance 
rp,,!!-, mialionshlp/pannershlps. 



[^5Mi^^i^i*?irt3S^anoe Improvements objectives 
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Avoid incoming quality inspecton 
Improvement in on-time delivenes 
■ Enhanced ioaistics support 

f Minnate goyemment -^^^^^^^i^^^ 

Resoiutionof these issues canbedifferent ford^^ 
rSd be addressed in the strategy at the business area level 



Business Area Evaluation Process 

leduping the total cost of doing business. 

These are major outcomes of a business area evaluation process: 



Mcasureinent 



Certification 



Improvement 



situations. 

^efr.rt ;c » rhallense to adopt the elements of the 
The business area evaluauon process effort « "^^f ^ ^ ^ Hate for your 

S2r-t^^^=lT=?-e„cea«/W 
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Bata/Camp/ 
Post 



Canlrai 



Commercial 



Noo-oommsfcial 



MEASUREMENT 



CERTIFICATION 



IMPROVEMENT 



SpaiBS 



Adopt the elements that are 
appropriate for the business area 




. systematicconectionofaccun.te,relevantdataforcontractorselection^ 

• Consistent approach to measurement across major business areas 

• Consistent feedback to contractors 

• Focus for supplier improvement 

• A tool for item level certification 

• A means to faciUtate benchmarking 

The major process elements for developing a measurement approach are: 



Develop Measurement Criteria 

Develop Approach for 
Data Valida tion b y Contractors 



Develop Performance Feedback Process 



. rrit^rin- F^ch business area's business area evaluation process 

common criteria for measuring supplier performance. Common req 
business unit include: 

. Development of a procedure detaiUng (1) rating frequency ; and (2) rating 
communication and use 

• Supplier Performance Report with ratings 

• Minimum reporting frequency of quarterly 
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• Distribution policy for rating results 

WMle i„co^«ti,.gcon™n criuriate^ca^ui^suppUcrpofon^^^ suppUers are 
rated on: 

. QuaUty of the products and services they provide 

• Delivery performance 

. Abmty to provide serto. including pertinent taformauo. 

Eachbusi«s,a.eam«s.aefinewhatcon«i».^P«^;^.«^°^^^ 

and service requirements. M--'^'P'7J^td^«^"'««l"^» 
defined tor tl« three rating -^-^ ^^^'^hS^s area «> devise the 

-^S^^a^S^'^tbSr^'nTw^----- 
Perforn^„«asuren..tis^dto^^co^^^^^ 

,0 ensure that contractors an= "^""^^^f'^Sormance in the execution of 

s^^n::^t^=r^-"-ss-»---'-«-'^^ 

^.vaUdationprocess^.«.;^y^^^c»^^ 

pertains to their perfonnance In the past, ^J^^^ . / performance 

^ addition, as infom^tion ntoves '^'^^ZZ^T:^^!^- 
from contractors that relates to their performance. 

appraisal. 

r ^A/TofrPror^M- Feedback is provided to suppliers on their 

^rce"srXts™:« 

may occur in any number of forms, includmg: 



♦K o..r.ni;/.« at least annually to infonn them of their 

improvement 

. More frequent notification by telephone or letter „,ative to 

with Other suppliers. 



3. Certification 

Creadon » *c i.«o,pa« or fanny of paj^Uvd ^ ^^"^ 

evaluation prcc«s. CcMfied i""^ P«-^ '^^1. for tmplytagl fom. fit 
subjected to incoaimgimp=cnon. ''"PP'*' Jf Certification is 

materials, assemblies, and printed matenals. 



Develop Certification Process Procedures 



Identify Key Strategic 
Critical Parts. Materials. A ssemblies 
Conduct Qua lit y System A ssessment 



Develop Total Cost Assessment 



points: 

• Item quality level 

• Fmancial requirements 

• Risk analysis of using a certified product 

• Supplier quality systems 

• Supplier process capability 
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• Item stability 

Procedures should also address: the sharing of information with all pertinent P^r^^s i_e., 
SsT™agement, other user locations;are^^^^^ 

X^on for certi/cation; and periodic audits or reviews to detenmne 
continuing certification, de-certification, or re-certification. 

Th. rriteria for and risks of certification will be determined at the immediate business 
A quality history must have been established for the suppUer facility P"'?"";^ ^^^^^^ 

the assessment must be resolved prior to certification of the item. 

weU as those that have the potential to reduce operating cost Part-level certification 
vital for base/camp and central commodities busmess areas. 

Conduct Quality System Assessment: The foundation for a contractor evaluation process 
!;Tqudity sys^n; assessment Assessment of a supplier's quality system can be 
viewed as a 6-step process. 
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Suppllar 
Quality 
System 
recognized 
in award 



Has is an example "Supplier QuaUty Process Evaluation Report" resulting from a 
quality system assessment. 



Supplier Quality Process Evaluation Report 



Supplto 



Facalmtt 



Evahiailon 



•Quality 
*Oocuin«nt 
•PufctMMd 
•Statistical 

•oaa* 

• Mattdal Control 
•Rnal 

•Conttnuouo 
/Cuatomor 



7 
9 

23 

1 

14 
22 
12 
12 
• 



TotriRiUna 132 



Max 

10 
10 
40 
IS 
20 
40 
15 
15 
10 



200 



Quiritty tmprov«(n«nt 

SkiniAcant Improvement has been 

Need to fo<£s on real ome statistical process oontrol. 



made In material control Inawslng emphasis on emptoyee training ^s 



This particular example includes both a quantitative rating and a narrative section to 
record identified improvement actions and other related remarks. 
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EUgibility for classification as a certified item should also include: 

• Responsibility for quaUty Ues solely with the suppUer of an item 

• Regulatory risk requirements must be reviewed to understand impact on the 
certification process 

. Financial risk consideration must be given to balancing the potential risks of not 
routinely inspecting items against total system cost 

Develop Total Cost Assessment: Effective total cost assessments, based on Activity 
Based Costing principles, are part of a contractor evaluation program. 

Total cost encompasses the "all in" cost of doing business with a suppUer. For example, 
acquisition cost, the cost of supplier's activities to process a customer order and 
suoDlier's materials plus profit, and consumption cost, the cost of a customer s 
activities-labor and overhead-to process a supplier's shipment through the customer s 
system. 

The objective of a total cost assessment, using Activity Based Costing principles, is to 
award contracts to the true lowest cost bidder. These are the issues to be addressed: 



IdentH 



itify historical nonproductive costs resulting from supplier 
nohcompliance 

• Use to estimate the tme cost or procurement bids 

• Suppliers' nonproductive events are "charged" in the program 



Quality Costs 



• Source rejection 

• Inspection resutxnittai 

• Return to supplier 

• Material review , 

• Shoo floor retactlon 
(latent defecs) ^ 

• Corrective action request letter 

• Supplier stop notice 

• CECA action 



• Schedule Costs 



• Earty . 

• Overshipment 

• Late receipt 



Stockless Production Cost 



• Inventory carrying 



Costs of Communications 



> Paper baaed or EDI 



Cost Containment 



• Stability of pridng 



Improvement: The improvement aspects of a business area evaluation process are 
aimed at evaluating contractor's progress in achieving the highest levels of performance. 
There are four major elements in the improvement process. 



Formulate Projects to Reduce C ostAmprove Qu'^ 
Develop On-Going Cycle of Continuous Evaluation 

& Improvement ^ 

Develop Improvement Recognition Programs 



Conduct Process Approvals: Hie scope of processes that are involved in iniprovement- 
rclaTed activities is more extensive than the certification process considers. Here the 
focus can extend to most of the following: 

• Quality/Service History 

• Cost Management 

• Environmental Initiatives 

• Quality Systems 

• Risk Management 

Additionally, management and technology factors are considered: 

• Management conunitment of the business area 

• Industry position 

• Technology position 

• Resource commitments to continuous improvement 



To the extent that on-going measurement and certification efforts ^^^^^^'^ f 
contractor processes, those processes that meet minimum requirements for approval wiU 
be approved. 

While not explored in depth, three possible scenarios for approving Pl^^'J^^^"^ 
rely on either third-party certifications, commercial certifications, or both (e.g ISO 
9000, Malcolm Baldrige National Quality Award criteria, or other ^or^^^c^ 
certifications); second, grant DOD certification; or third, either include th^d party 
commercial DOD, or aU of these at the time of each acquisition-not m advance as 
Z^^To.l Tlis would potentiaUy include ISO 9000, Baldrige, or other commercial 
certifications in addition to DOD. 

For items used in private industry for which DOD has a n^d, third party or commer^ 

c^fications may be appropriate. THe administrative burden 

minimal in this scenario. In other situations, where the item is unique to DOD, DOD 
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criteria may be appropriate, 
to administer. 



This scenario may require a greater burden-and thus cost- 



to achieve world-class performance. 

^^.^r:^Tor^ in addition .c ^ certiflcaaon 
previously achieved. 

formulated and progress tracked and evaluated. 

Tl» best cost reduction and quality improvement results will b« obtained from steady, 
focused continuous improvement. 



. time and resources to target orolects that can significantly reduo 

improvement 



Managing value into both part fs 
operafions on an ongoing basis . 
A successful program incorporates 

^*^®S"PP'!.®&^agement commitment 

- Quali^/Service history 

- Cost management 

- Geography 

- Environmental initiatives 

- Quality systems 

- Industry position 

- Technology position 

- Risk management plans 
. Resource commitments 




Eliminate Scrap , 
Optimize Packaging 
Increase Delivery 

PaSeSvofk/Xdim. Cost 

Reduction 
Inventory Turns & 

Investment . 
Product Design Review 
Process Reviews 
Supplier Devetopment 

Deeper into the Suppty 

Chain 



The respective DOD business areas should be proactive m aU aspects of the busmea 
liSon process. b«t panicuiarly when undettaking 
^e^ves. The "panneriBg- tnodel is the posture that should be used to gmde 
contractor imet&ces in the improvement environment. 

Develop ImpravemM Recognition Program: A propam *<'"l'',?«„''r''^^^ ^ 
rimvi/feedbadc and to recognize accomplishments. Documentation that descnbes the 
Ct^v^^on prS»s and ob^es should be amiable for aU mterested 
cTJ^,^. Periodic meelp and reports should be a part of the p«.gram. 

The cost aspects of the contractor evaluation process track with the level of contractor 
evaluation the business area has adopted. 

• Measurement-Cost Assessment 

• The minimom needed to support best value 

• Certification-Total Cost Assessment 

. Encompasses the "all in" cost of doing business with a suppUer 

• Improvement-Cost Reduction 

' . Focuses on specific target opportunities to reduce cost using acuvity 

based costmg 



Activity Based Costing Process Flow 

Pli^fHufftg or Tools Ugetf 



^^^^^ °^sus^e6m^<»^or 




norvvalue^dded activities 



Interviews, observations, personal knowledge. 
Ftow charts 

Data Collection Sheets 
Cost Model Woflcsheet 
Cost Wortcsheet Summary 
Procedure/policy changes 



Here's how industry suppUer evaluation programs and existing Government past 
p^o^rWoLion systems align with die Contraaor Evaluauon Program. 
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Improvement 



Grainger Distributors 



Blaclc&Decker 



Mobil Oil - R J. Reynolds 



a far lesser degree or not at an. 



Certlflcatlon 



Measurement 



Business Area Analysis 
contractor EvaluatSS? program Strategy 



..„h.,n.Re,u.re^—Con«c.or.va.ua«o„ program 

conducted in two parts. 

The fustpan was to ?c^<>^^'^'^f^Ztic^^^^'^^ , 
^ ;^™Fvaluation Program model to identify specmc i identified 

fl« functional req—^ fa 
Program model. The method w 
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• Identify from the Contractor Evaluation Program model additional functional 
and performance capabilities that will be required to satisfy new or changed past 
performance requirements 

• Identify from the Contractor Evaluation Program model functional and 
performance capabilities that proivde opportunities for increased economy and 
efficiency 

We implemented a Use Case Approach to analyzing and documenting functional 
requirements for the Contractor Evaluation Program model. Through Use Case 
Analysis, we divided the Contractor Evaluation Program model into a collection of use 
cases. Next, textual descriptions of use case were developed to describe the graphical 
information presented in use cases. 

Once the Contractor Evaluation Program fimctional requirements were documented, we 
compared them to the functional requirements for the Standard Procurement System. 
We analyzed the Standard Procurement System functional requirements related to 
colUcHon of contractor past performance information. They were found primarily in 
two areas: 

Under Administer Contract, it indicated that the system shall: 

• Notify the user when previously-identified criteria for contractor performance 
have been breached 

• Process material review board actions and corrective action requests/ notices/ 
plans 

• Track contract performance reports 

• Notify the user when performance parameters do not meet user-defmed criteria 

• Prtocess shipment and performance data against the MILSTRIP requisition 
number and contract schedule. 

Under the Procurement Planning functional requirements, it indicated that the system 
shall perform a Contractor Assessment. In doing so, the system shall automatically 

• Aggregate contract performance information into contractor summary 
performance reports 

• Use these summary reports along with other contractor information to create 
vendor rating summary reports. 
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We also analyzed the Standard Procurement System functional requirements related to 
use of contractor past performance information. They were found primarily in the 
Solicit Offers and Award Contracts area. 

]n the SoUcit Offers and Award Contracts section it indicated that to evaluate offers, the 
system shall provide the capability to: 

• Evaluate offers based on the offer data and previously-defined criteria 

• Integrate offer data and previously-established evaluation criteria to perform 
evaluation 

• Integrate an offeror's past performance information into the evaluation process, 
and recommend a determination of responsibUity based on user-defined criteria 
and algorithms implied to previously entered data 

• Be able to create, request, receive, and dispose of pre-award survey requests. 

Our analysis showed that the Contractor Evaluation Program model functional 
requirements arc consistent with apparent Standar Procurement System contractor past 
performance functional req'zirements firom the standpoint that use and coUection cnten 
are user-defined to the Standard Procurement System. 



4. System/ Process Issues for the Contractor Evaluation Program 

The Contractor Evaluation Program implementation must consider the following 
system/process issues. The general criteria for their appUcation is defined in terms of 
which of the three sections of the Contractor Evaluation Program the specific 
system/process relates-Business Area Plan, Business Area Strategy, Business Area 
Evaluation Process. In two cases-rFaimess and Due Process~the issues appeared to be 
more related to the Government's conduct of the contractor selection process and how 
the information would be used than the attributes of the Contractor Evaluation Program. 
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GENERAL CRTTEWA FOR APPLICATION 



Issues 


Business 
Area Plan 


Business 

Area 
Strategy 


Business 
' Area 
Evaluation 
Process 


rentralized / necentralized 
The degree and level of centralization. Is data 
aggregated to the product center, major 
command or HQ level? 




X 




Automated / Manual 

To *tto mct^m fir nrnce5;^ automated, seini- 
automated or manual in the manner in which 
past performance information is collected, 
maintained and disseminated? 




X 




rnnfidentialitv 

System's capability to protect, limit, and 
otherwise effectively control agamst 
unauthorized access to contractor past 
performance data. ■ . 






X 


Data Availability 

System's capabUity to rapidly disseminate the 
requested standard and tailored information on 
real-time or time delay basis. 




X 




Turrencv / Tntftpritv / Accuracv /Validity 

- System's capability to present latest relevant 
information, upcate ana purge uaux, «mu uh.^ 
period covered 

- Can the data sources be identified and are they 
appropriate? 

- The system's capability to provide complete, 
comprehensive, validated data. The system's 
capability to align past performance evaluations 
with sources 

. Has the data been validated by internal and 
external sources as appropriate? . 






X 
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Source and Tvpe of Data 

Categories of data included in the system — 
government, DoD only / commercial. 
Quantitative, qualitative data. Data trends. 
Comparison data. 




X 




Mergers and Acaui^itions 

System capability to report on company*s past 
performance that occurred prior to a merger or 

acquisition. 






X 


Subcontractor Involvement 

The system s capabuity to discern between the 
prime and its subcontractor's past performance 
on prior contracts 






X 


Fairness 

The system s capability to treat all offeror s 
equally and ensure past performance data is 
evaluated with the same impartiality as other 
evaluation data 






Contractor 
Selection 
Issue 


Opportunity for contractor to respond to 
weaknesses / deficiencies documented in the 
governments evaluation process 






Contractor 
Selection 
Issue 


Lack of Past Performance Historv 




X 




The system's capability to overcome / handle the 
lack of past performance data for a particular 
contractor. 


Threshold of Applicabilitv 

The system's capability to apply data according 
to cost, time, or other (dollar) thresholds. 






X 


Capability of Attribution 

The system's capabihty to shield the sources of 
data from unwarranted disclosure. 






X 


Consequences to the Contractor 




X 




The system's intended / untended penalties / 
rewards for poor / superior past performance 
data 
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Feedback 






X 


Recoenition 




X 





The system/process issues that should be considered under the Business Area Strategy 
aspects of the Contractor Evaluation Program are: 

• Centralized/Decentralized 

• Automated/Manual 

• Data Availability 

• Source and Type of Data 

• Lack of Past Performance History 

• Consequences to the Contractor 

• Recognition 

The system/process issues that should be considered in the Business Area Evaluation 
Process aspects of the Contractor Evaluation Program are: 

• Confidentiality 

• * Currency/Integrity/Accuracy/Validity 

• Merges and Acquisitions 

• Subcontractor Involvement 

• Threshold of Applicability 

• Capability of Attrition 

• Feedback 

Specific criteria for application of these issues should be developed as part of the 
respective Business Area Strategy or Business Area Evaluation Process considerations 
to which they relate. 



SECTION III. D. Business Case Analysis 



This section provides an analysis of three alternative approaches to past performance 
poUcTimpleLntation. TTie three alternative models that will be used for companson 
Tu^s'S ti.e business case analysis are the "As-is" model, wMch is s^ctur^J^m 
Sormation in Section H. A.; die DFARS model developed from ^^^^^^^ 
FAR and die proposed changes to the DFARS; and the ContractorEvaluation Pro^ 
moderthe To-Be model developed from information in Section HI C. In each niodel a 
"Sti^n is made between the "Collection of Past Performance Information for Future 
UsTtd the XoUection and Use of Past Performance Information dunng Contractor 
Selection". The analysis of each model focuses in these two areas. 



1. As-ls Model 



The As-Is model for existing government past performance information systems as 
depicted below, provides a top-level flow diagram for tiie sitiiation pnor to recent 
cS^gestotheFXRandtiierelatedDFARScase. -^-^-^^^^^^"t^'^'ZZde 
die coUection of contractor performance information for future use. The nght-hand side 
depicts die principal activities performed for die coUection and use of past performance 
information during contractor selection. 
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I Collectton of Past 
Perfonranoe 
Information for 
Future Use 



by 
tor fciw 



ffYSTgMATIC 
. jv^utfion Mtd voidalion a4 this 
tor lutum uM wM «tart«d in Mvw«t 



»A^E ai ** u M m l T i n qACASS 
• Commjdton utftzing CCASS 



RYQvidABVM 
• Aaraiwulla 



CPARS 



AanningitM pMtoiranos of 



«iitanttM.but. 

J^iwiy tor ad hoc «• tod wiftou 
i lomirt i»-*wr «1 fObi«id p«r— 
hwoM^^weortrMB^ 




There are two approaches to coUecting past performance information ^^^t^f^^' 
Thighiighted on the left-hand side of the diagram - ad hoc and J^^^ 
hoc coUection of past performance information for fiiture use has been, and contmu^to 
braltinepracaTbrsomeprocurementauthorities. However, the pnmary purpose 
has been to support local contractor selection decisions. And m most cases the 
p^rfoLnce evduations are not provided to contractors for review and PJ-^^^^^^' 
knd the evaluations are not identified as "source selecuon mformaUon and ffled for 
possible use in the fiiture. Information from thesfe activities is used m con^tor 
Son decisions, as depicted on the right-hand side o^^^^f^f^J^^l^^:^ 
other past performance information that may be gathered at the ume of a contractor 



selection decision. 



The systematw approach for galhertag contractor performance mfonnanon on acuve 
contS:ts is used to the systenB that are identified and descnbed in Sccuon m. /V of dus 
"p^. The« existing systenis operate in essentially two different ways One^heson 
the existence of petfomance tncbng data at the contract bne iten, level. Th«e d^ are 
used to calculateVformance ratings based on previously established ~ nUes. 
For reference purposes, we have classified these systems as perfonmnce traclang 
ZTT^P^y *eU systems address attributes of supplies and equipment Aat aje 
S^bleL delected at the time of. or subsequent to deUvery by the contractor. The 



principal focus of U,es= sy..,ns is '-"^^^''^^^^^'t^'^^':^ 
in Appendix A- 

I^eother type of systemfor gathering contrac^P^^^^ 

appraisal of a contractors performance ^^^^^^,^Z7o± performed on 
of the work performed by the ^^^^/j^P^^^^^^ classified these 

the total contract or contract order For ^^^^I^^PT^^; Systems address not only 
sy^^^r^ ^sperfornuince appraisal systerns.Tmc^^^^ 

tie quality and timeliness of products f ^^^^^^^^ ^^^^^^ technical, 
factors dealing with the perform^ce of ^^^^ "'"P^^^^^ f^to« might include anyone or 
cost and schedule performance of ^^^-^^^^^^l^^^^^ and the nLe of 
all of the following, dependmg on the circumstances oi inc 
the product or service that b being acquired: 

• oyenuns experienced on reimbuisable contracts, 

• responsiveness to technical (Erection, 

• effectiveness in numaging the provisions of the contract. 

^ienKMit, quality assurance, continuous process improvement, etc.) 
. the quality and thoroughness of research conducted under the contract. 

« , • nnfi^HTPARS ACASS and CCASS as the existing systems that use 
analyses for these systems are included in Appendix B. 
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products or services, but there is no stated requirement to address the approach for 
dealing with contractor past performance as a part of these strategies and plans. 

The activities at the bottom right of As-Is Model diagram are performed in connection 
with gathering, validating and using past performance information at the time of a 
contractor selection decision. These activities are: 

• veiify past pcdbrmance infonnation fimiished by contptctois in response to the requirements in 
the solicitation; 

• access validaied past perfonnanceinformatioa for use in soun:esd^^ 
may exist; and ^ 

• gather and assess past performance information from other sources, sudi as inforaiatioD 
available from risk assessments, process reviews, government maintaine d databases, 
performance award listings, and commercial survey services. 



The final step in this process is the utilization of past performance information in 
contractor selection decisions based on the ground rules established for the acquisition 
and consistent with the evaluation criteria and other information provided to the offerors. 



B. DFARS Model 

The DFARS model is depicted in the diagram on the next page. It is based on the our 
interpretation of the DFARS Case at this point in time. With respect to the collection of 
past performance information on active contracts for future use (as portrayed on the left 
-side of the diagram), the key differences from the current model (discussed in the 
preceding paragraph) are that: performance evaluations will essentially be required on 
all contracts above $100,000 with few exceptions; and, results of the evaluation will be 
provided to the contractors for their review, comment and possible rebuttal. Methods 
for handling the review process and resolving any differences between contractor and 
government officials are also covered of the model. 



With respect to the collection and use of past performance information at the time of 
contractor selection, as shown on the right-hand side of the DFARS model, the key 
difference is the addition of a requirement to use contractor past performance 
information, except in those cases where its use is not found to be practical or useful. In 
those cases, the contracting officer must document in the contract file the reasons why 
past performance was not used. 
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Collection of Past 
Perfonnanco 
Information for 
Future Use 
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prHCrfbad «or uaa by tha <MI 
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'CflfMHOMOttC .tMamountof 
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anOy Of nOI ^IfaKHIV 
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Collection and Use of Past Performance 
Information During Contractor Selection 



andanalyali 



Davatap acquiattkxi 
ttmagiaa and plant 
tot proourafnant 



paflonnanoa prawtalona 
for induiiona In 



Qatharanda 



No 



SAME AS THE 
*AS-tS'M00e. 



SpacMc fapuiramanta 
, hawa baan attabSihad 

tar uaing oontrador pait 



In itm aourea ■aiaaioo 
pTDoaaa. II not uaad* 



>:-x-x-:«««:->K««««c*»:«-x«^^^^ 



A significant point concerning the DFARS model is that Although the FAR states that 
the requixement for the evaluations should be tailored to the size, content and 
complexity of the contractual requirement, the DFARS (as of the current draft) 
establishes standard evaluation criteria and common data elements to be used in all 
cases. Consequently, the amount of "tailoring" could be constrained by the use of the 
standard criteria. The following table provides a summary of these provisions of the 
FAR that indicate that the collection and use of past performance information can be 
tailored to the particular circuxnstances of the procurement. 



Collection of past performance 
information for future use 

(Ref. FAR, SUBPART 42.15) 


Collection and use of past performance 
Information at the time of source selection 

(Ref. FAR, PART 15) 


The content and fomiat of 
performance evaluations shall be 
established in accordance with 
agency procedures and should be 
tailored to the size, content and 
complexity of the contractual 
requirements. 
[Ref. 42.1502 {a)I 


The cognizant technical official is responsible for the technical and 
past performance requirements related to the source selection 
process. [Ref. 15.604(b)] 

Past perfonnance shall be evaluated . . . unless the contracting officer 
documents in the contract file the reasons why past perfomnance 
should not be evaluated. [Ref. 15.605 (b) (1 ) (11)] 
The source and type of past perfonnance Information to be included in 
the evaluation is within the broad discretion of agency acquisition 
officials and should be tailored to the circumstances of each 
acquisition. [Ref. 15.608 (a) (2) (li)] 
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The FAR provisions provide government officials the latitude to develop and adopt a 
tailored approach that fits the specific circumstances of each acquisition. However, the 
FAR does not go into the nature and extent of the tailoring that is envisioned. 

On the other hand, the proposed DFARS provide criteria and a rating scheme that shall 
be used in the evaluation of contractor performance. These prescriptions could be 
construed to establish boundaries around the degree of tailoring that would be 
acceptable. 



The evaluation criteria and rating scheme in the DFARS is summarized below. 





Ratings 




Qualrty of Product or Sendee (a required element). This Inctudes the 
followTng aspects of performance: 

1. Compiiance with contract requirements; 

2. Accuracy of reports; 

3. Appropriateness of contractor personnel assigned to the contract 


0 Unsatisfactory : Nonconfomfiances compromise 
(or are compromising) the achievement of contract 
requirements, despite the use of Agency resources, 

1 Marginal : Nonconformances require major 
Agency resources to ensure achievement of 
contract requirements. 

^ oousrociory . NorK^ontormances do not Impac^^ 

achievement of contract requirements. ' 
3 Excellent : There are no quality problems. 




Co«t Control (not requiiBd for fInn-fixediJrice and flrm-fixed-price with 
^ economic price ad)ustment contracts). This includes the fdliowing aspects 
of perfomiance: 

1. Current accurate, and complete bIDlngs; 

2. The relationship of negotiated cost to actuals; 

3. Cost containment initiatives; and 

4. The number and cause of change orders issued. 


0 Unsatisfactory : Cost Issues are compromising 
performance of contract requirements. 

1 Marginal : Cost issues required (or require) 
Agency resources to ensure achievement of 
contract requirements. 

2 Satisfactory : Cost issues do not impact 
achievement of contract requirements. 

3 Excellent : There are no cost issues. 




Timdinesa of Performance (a required element). This includes the 
following aspects of performance: 

1. Whether the contractor met Interim milestones; 

2. Contractor's responsiveness to technical direction; 

3. Contractor's responsiveness to contract change orders and 
administrative requirements; 

4. Whether the contract was completed on time, including contract close- 
out and reporting responsibilities and contract administration; and 

5. Whether liquidated damages were assessed. 


0 Unsatisfactory: Delays are compromising the 
achievement of contract requirements, despite the 
use of Agency resources. 

1 Marginal: Delays require Agency resources to 
ensure achievement of contract requirements. 

2 Satisfactory: Delays do not Impact achievement 
of contract requirements. 

3 Excellent: There are no delays. 




Contracting / Business Relations (a discretionary element). This 
includes the following aspects of perfonnance: 

1 . Whether the contractor effectively managed the contract effort; 

2. How responsive the contractor was to contract requirements; 

3. How promptly the contractor notified the Govemment of problems; 

4. Whether the contractor was reasonable and cooperative; 

5. How flexible the contractor was; 

6. Was the contractor proactive; 

7. How effective were contractor recommended solutions: and 

8. Did the contractor effectively Implement socioeconomic programs 
including compliance with requirements of the clause of FAR 52.2*1 9- 
8, Utiltzation of SmaU Business Concerns and Small Disadvantaged 
Business Concerns, and 52.219-9, Small Business and Small 
Disadvantaged Business Subcontracting Plan. 


0 Unsatisfactory: Response to Inquiries, technical 
service, and administrative Issues Is not effecth/e 
and responsive. 

1 Margirial: Response to inquiries, technical 
service, and administrative issues is marginally 
effective and responsive. 

2 Satisfactory: Response to Inquiries, technical 
service, and administrative issues is usually 
effective and responsive. 

3 Excellent Response to inquires, technical 
service, and administrative issues is effective and 
responsive. 
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There may appear to be some advantages in a single set of evaluation criteria for all 
contracts, just as there are apparent disadvantages that bring into question a "one size 
fits all" approach to evaluating contractor past performance. These questions are 
addressed later in this section. 



C. Contractor Evaluation Program Model C'To<be" Model) 



The diagram below provides another view of the Contractor Evaluation Program 
described in Section HI. C. The principal difference between this model and the As-Is 
and DFARS models is the business area focus shown in the shaded area in the center of 
the diagram. This emphasis on business areas includes a business area plan, similar to 
the market research currently addressed in the FAR but broader in scope; business area 
strategy to guide the collection and use of past performance information; and a business 
area evaluation plan, tailored to the specific requirements of the business area. 



I Collecttonof PMt | 

PorfomuinM 

|: Information for | 

Futura Uaa | 



tni (ito Monviition 
torfutunum 



' Develop a 
business area 

plan 



Develop business 
area strategy 



■ 



Develop and 
implement business 
area evaluation plan 



Coilai^on aiid Uaa of Paat Parformanca 
Information During Contractor Salactlon 



Conduol 



Qiiidiln— taf gnmta hxvtiont 
0 on Bu«nM« arM anaiyvt. 



Qatfwr 



UtiUzepASt 

Mormatton to mafcs 
baa value decwkint 



The principles that were used in designing the model were derived from an analysis of 
our previous research in this area and the government and industry approaches to 
contractor past performance and supplier evaluation we reviewed in this study and 
included die following: 

• A cost-effective ^proach to the collection and use of contractor past performance information 
depends on, and is sensitive to factors related to the business areas in which products and 
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services arc procured and used (as opposed to a universal approach that can be applied to the fall 
range of products and services procured by DOD in all sectors of the industry). 

• A business area consists of a homogeneous group of products or services which share similar 
characteristics and fcr which a forward-looking plan and a coherent and congruous strategy and 
evaluation process can be developed. 

• Business areas can be local or extended in application. In their most robust form, they constitute 
the horizontal integration of products and services. 

• The process for in^ilcmcnting contractor past performance issues in a particular business area is 
developed firom business area plans and strategy for the specific business area and typically 
involves a cross-functional team effort 

• The initial and vital step in developing plans and strategy for a business area is an analysis that 
covers the requirements for the product or service, past and projected; the industry composition 
and basis of competition; and the market trends and specific performance of leading companies 
in the industry. 

• The business area plan and strategy will provide the basis for developing a tailored approach to 
the collection and use of contractor past performance information in the particular business area 
as well as the foundation for a total program designed to incorporate best value practices into the 
procurement process and to attract contractors and suppliers committed to high levels of 
performance. 

• Information technology wiU be utilized to facilitate communication between Government 
managers in separate organizations with a need to share infonnadon about business area 

, strategies and plans as well as the past performance of individual contractors in those business 
areas. 



The principle elements of the Contractor Evaluation Program are: 

• a business area plan; 

• a business area strat^ in the context of an overaU acquisition strategy for the business area; and 

• a business area evaluation plan diat can be used to tailor the contractor past performance 
provisions of the FAR and the DFARS to the particular business area. 



Each of these elements is interrelated. The model's three major components are shown 
at three levels of detail in the diagram below. 



The business area plan, in the top center of the diagram, includes a definition of the area 
to be investigated; a research program into the area, including the internal requirements 
as well as the industry sources for the particular product or service; and a comprehensive 
analysis of all relevant factors dealmg with doing business in the particular area from the 
perspective of the government acquisition officials. 

The business area strategy, shown on the left-hand side of the diagram, encompasses 
three principal focus areas. These include: 

• intei^tion of contractor past performance issues with the broader issues related to the overall 
acquisition program for the business area, and with the cross-fiinctional considerations that may 
be involved (e.g., cogineering, test and evaluation, production, logistics, risk management, and 
quality); 

• the goals and objectives for the contractor past performance program in the business area, 
including the desired level of performance sought from contractors with whom contracting 
relationships exist or are anticipated; and 

• an overall approach for the use of past performance information designed to achieve the past 
performance goals and objectives established for the business area, and to provide guidelines for 
developing a cost effective plan tailored to the particular business area. 

The purpose of the business area evaluation process, as depicted on the right-hand side 
of the diagram, is to lay out a plan for executing the business area strategy. Three 
principal area are identified for possible coverage in the plan, although others may be 
added when warranted in a specific business area. Whereas all plans should address the 
performance measurement aspects of the program, the other two areas will be covered to 
the extent that the agreed to strategic approach provides guidance and direction in these 
area. The three principal areas include: 
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• A plan of action to measure the performance of contractors based on criteria tailored to the 
business area and consistent with the strategy for the business area; 

• A plan of action to certify the performance of contractors consistent with the strategy for the 
business area and that considers the certification of processes, products and services (e.g., "Blue 
Ribbon** Programs); and 

• A plan of action to undertake initiatives designed to improve the performance of contractors that 
constitute the supplier base, and in a manner consistent with the strategy established for the 
business / program area {e.g., process improvement initiatives and recognition programs). 



The third tier of the diagram is intended to address the methods that will be used to 
collect past performance information on contractors that participate in the business area. 
The following methods are candidates for use: 

• A continuous measurement program that may include provisions for review of the information 
by the contractor, a process for dealing with the resolution of contractor rebuttals; and the 
maintenance of the informadon for ftiture use when needed for source selection purposes, or 
some other purpose consistent with the strategic plan. This approach includes: 

• Performance appraisals, on a periodic basic at the contract, or contract order level, based on 

an assessment by the responsible government official.(s) for contract technical and /""^ 
management oversight; and ^ t 

• Performance tracking, on a continuing basis at the contract line item level, based on quality 
■ and delivery data collected as a part of established contract management and oversight 

processes. 

• An ad hoc measurement program that is designed to provide past performance information when 
needed to support contractor selection decisions, or some other purpose established in the 
strategic plan. This type of program may include information from sources such as: 

• Surveys of prior customers-c.g,, reference checks; 

• Requests for past performance information from contractors (e.g., in response to 
solicitations); 

• On-site assessments of contractor operations to include their technical and management 
processes; and 

• Information gathered from other available source (e.g.. product performance and reliability 
data, CSCS data, certifications and awards, etc.). 

The Contractor Evaluation Program is a comprehensive approach for collecting and 
providing information on the past performance of contractors for contractor selection 
purposes. It is also an orderly approach to tailoring the policy and requirements on this 
subject to specific business areas, and to achieving and sustaining improvements in the 
overall level of performance that is exhibited by contractors in the business area. 

Difference Between Contractor Evaluation Program and Proposed DFARS 
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The following summarizes the principal differences between the Contractor Evaluation 
Program and the DEARS model in dealing with contractor past performance issues, 
policies and requirements. Each of the criteria indicated in the chart is discussed in the 
following paragraphs. 



CRITERIA 



FLEXIBIUTY 

Organizations are provided latitude and empow - 
ered to tailor requirements and guidelines to fit 
particular characteristics of their business areas 

SCOPE 

Consideration of past performance Infomiation 
extends beyond its use in source selection 
decisions to Include improvement initiatives 

BUSINESS AREAS FOCUS 

An analysis of business areas is recognized as a 
key factor in developing effective strategies and 
plans for deafrig with past performance issues 

PROCESS IITTEGRATION & TEAMWORK 

Past perfomiance 'Strategies and plans are 
developed consistent with overall acquisition 
strategies and utilizing cross-functional teamwork 

VALUE TO THE USER 

Information on the past perfonmance of contractors 
provides a valuable input to sources selection 
decisions and is shared with other organizations 

SHARING INFORMATION 

Provisions ara made for sharing contractor past 
performance information among DoD 
organizations in a cost effective manner 



DIFFERa^CES 
DFARS TO-BE MbOEL 



Limited 



None 



No 



Limited 



Limited 



Not addressed 



Substj ntial 



Signifi :ant 



Yes 



Yes 



Subst< ntial 



Yes 



Flexibility 

The flexibility criterion addresses the capability to deal with, and adapt to the particular 
circumstances of an acquisition progranL It is especially important in the 
implementation of past performance policy and requirements within the DOD because 
of the wide range of products and services that are acquired and the wide range of 
circumstances that may affect the acquisition process leading up to the selection of 
contractors and to the award of contracts. In addition, the post-award activities and the 
contract management approach employed by DOD components and agencies arc also 
subject to considerable variability depending on factors such as the size, scope, 
complexity, and nature of the contracted work. 
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Both models are designed to address the flexibility criteria. However, the extent of the 
flexibility in the Contractor Evaluation Program is considerably greater than the FAR / 
DFARS model as summarized in the following table. 



FLEXIBILITY DIFFERENCES 


CONTRACTOR EVALUATION PROGRAM 


FAR /DFARS MODEL 


Organizations with contracting authority and 
technical oversight responsibilities shall establish 
the content and format of performance evaluations 
based on analysis of their txjsiness areas arKi 
consistent with strategies and processes 
estabfished by these organizations. 


Content and format of performance evaluations 
shall be established in accordance with agerx:y 
procedures and should be tailored to the size, 
content and complexity of the contractual 
requirements. (Ref. FAR 42.15) 


Contractor evaluations will be tailored to the 
business areas in which the contracts are issued 
and to the requirements established in the 
contracts, and this tailoring may extend to the 
thresholds used and to the provisions for review 
and rebuttal by the contractor. 


Contractor evaluations will be prepared on all 
active contracts at>ove the $100,000 threshold, 
reviewed by contractors subsequent to the 
evaluation, and filed and protected as "source 
selection information' after resolution of any 
rebuttal by the contractor (Ref. FAR 42.15) 


Principal users of contractor past performance 
information irxdude government officials irrvolved in 
contracting dedsions. and therefore these user of 
the information shall have a major role in 
determining the scope and content of contractor 
evaluations in specific business areas. 


The evaluation of contractor performance shall 
include specific data elements and evaluation 
areas, factors and ratings (as delineated in the 

proposeo UrAno lOj 


Same as the FAR / DFARS model, except that the 
cognizant technical officials will also ensure that 
their responsibilities are discharged in a manner 
consistent with the strategy and the plan 
established for the business area. 


The cognizant technical official is responsible for 
the technical and past performarx:e requirements 
related to the source selection process. [Ref. FAR 
15.604 (b)] 


Same as the FAR / DFARS model, except that the 
contractor past performance strategy and 
implementing process shall cover guidelines and 
provide decision rules for determining the 
inclusions of past performance factors in source 
selection and contracting decisions. 


Past performance shall be evaluated (in contract 
award decisions) . . .unless the contracting officer 
documents in the contract file the reasons wtiy past 
performarKe should not be evaluated. [Ref. FAR 
15.605 (b)l 


Same as the FAR / DFARS model, except with the 
addition that the source and type of past 
performance information is first tailored to each 
business area and the approach is described in the 
strategy for the business area. 


The source and type of past performance 
information to be included in the evaluation is 
within the broad discretion of agency acquisition 
officials and should be tailored to the circumstances 
of each acquisition. [Ref. FAR 15.608 (a)] 



A more rigid structure for the collection and use of contractor past performance 
information would very likely simplify the information processing functions and the 
automated systems that may be used to support these functions. However, our analysis 
indicated that the value of past performance information to the user for contractor 
selection purposes is diminished as the degree of standardization is increased in the 
evaluation process and in the collection of the information. And the views of many of 
the government officials who were interviewed during the course of the project and who 
participated in the workshops that were conducted, tended to support this analysis. 
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Scope 

The scope criterion is intended to address the coverage provided for all types and 
sources of contractor past performance information as well as for all potential uses of 
this information beyond its use for contractor selection purposes. 

Other than one statement in Part 15 of the FAR, the current provisions of the FAR, as 
supplemented by the provisions in the proposed DFARS, essentially address one type 
and source of contractor past performance information (in Subpart 42.15), and one 
purpose served by this information (i.e., for source selection purposes). The one 
statement in the FAR that addresses other types and sources of past performance 
information s^pears in Subpart 15.8 on the subject of proposal evaluation. 

... the solicitation shall afford offerors the opportunity to identify . . . contracts performed by the 
offerors that were similar in nature to the contract being evaluated* so that the Government may 
verify the offerors' past performance on these contracts. . . . Past perfoimance information may also 
be obtained from other sources imown to the GovenunenL The source and type of past performance 
information to be included m the evaluadoa is within the broad discretion of agency acquisition 
officials and should be tailored to the circumstances of each acquisition. Evaluations of contractor 
performance prepared in accortiance with Subpart 42.15 are one source of performance in.fomiation 
which may be used. 

Whereas supplemental guidance could be provided to address the other types and 
sources of past performance information and the potential use of this information for 
other purposes, there is currently no final version of this guidance. The OFPP guide on 
best practices for past performance, published in May 1995, is recognized to be an 
interim measure. And some of the guidance provided in this document does not appear 
to have universal relevance to the DOD procurement program. ^ ^ 

The Contractor Evaluation Program is intended to provide a broader perspective to 
contractor past performance and is designed to address a total systems approach to the 
collection and use of past performance information, to include: 

• The type of analyses required to develop a tailored approach by business area; 

• A definitive strategy for dealing with the entire issue of contractor past performance m each 
business area in a manner consistent with the overall acquisition strategy and procurement 
planning for the business or program area; and 

• A process focused on the actions necessary to execute the contractor past performance strategy in 
areas such as performance measurement, product and process certification, and performance 
improvement initiatives. 
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Coverage in the Contractor Evaluation Program is provided not only for the 
performance appraisal information that is addressed in Subpart 42.15 of the FAR, but 
also performance tracking systems, such as the Navy's Red/Yellow/Green system and 
DLA's Automated Best Value ModeL In addition, the proposed model covers past 
performance information related to the certification of contractors for products and 
services as well as the processes employed by the contractors. 

Business Area Focus 

This criterion addresses the capability to effectively deal with the size, scope and 
diversity of the DOD acquisition program. 

The proposed Contractor Evaluation Program recognizes that the products and services 
acquired by DOD span everything from sophisticated, multi-million dollar weapon 
systems to relatively simple, inexpensive commodities. Using FY *94 data, the 
following table illustrates the size of the DOD procurement program as well as the range 
of products and services that are acquired. Also shown is the breakdown of the total 
dollars into the various categories. 
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FY94 funds in SM 


Money 
Spent 


% 
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21,82 


41 0< 


A^Cornminitv Service 


1.7 
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id Space Vehides 



id Ships. Small Craft, 



2d Ship and Marine Eqtipment 



2^ Railway Equipment 
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24TractOf5 
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2qTtres and Tubes 
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7d Books. Maps, and Other Publications 
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9d Miscellaneous 
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Included in the tabulation shown in the table above are all contract actions above 
$25,000. The total of these actions was about $118 billion in FY *94, which was 



divided into about 19% for R&D, 37% for services and construction, and 44% for 
supplies and equipment And the total DOD procurement program for FY e94 
accounted for about 67% of the total for all federal departments and agencies. 



Also relevant to fully grasping the size and scope of the DOD procurement program are 
the number of organizations that have procurement authority and technical oversight 
responsibilities for a portion of the total program. Some of the principal organizations 
in each militaiy service and DLA are listed below. In addition, contracts are awarded by 
the operational organizations in each service including bases, posts and camps. 
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CofMracdnQ 



- Oanarai Supply 
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Caftar. Oayton 

~ Fual SuppV CaoMr. 

VA 



-ki^flMH Supply 
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'PafBonnal Si^port 



Pacflc Aiamada 

"ADPfTCanmamQ 
Offtoa.Ateandrta 



Canar. Art. VA 



Prindpal Orsamzatioos That Acqtiire Products and Service 

' — . ' When viewed in its totality, the DOD acquisition program dwarfs anything in the 

commercial world. Even the largest commercial operations are relatively small by 
comparison. And most of the major firms focus their business in a relatively few areas 
(e.g., automobiles, software, aircraft, etc.). 



The tremendous size, scope and diversity of the DOD acquisition program, as indicated 

in the preceding discussion, represented a significant challenge during the course of 

conducting the study and examining the contractor past performance issue. It was found 

that most any discussion or analysis of contractor past performance required a ^"^^ 

qualifying statement that established the particular segment of the total program that was 
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being addressed. For example, a method that was reasonable and logical for one segment 
was found to be irrelevant or even counter productive in other areas. These observations 
provided the basis for one of the design characteristics that has been built into the 
Contractor Evaluation Program — that is, the capability to deal with the inherent 
differences in the various segments of the DOD acquisition program. 

The business area focus in the proposed model is achieved by addressing each business 
area as a separate entity, analyzing the factors relevant to the business area, and then 
devising a strategy that makes sense for dealing with contractor past performance at the 
business area level. And this information is then used to develop ground rules and to 
devise a process, not only for considering past performance in contracting decisions, but 
also for improving the overall performance of contractors in the particular business area. 

The FAR / DFARS model does provide for tailoring but does not address the idea of 
using the business area analysis as the basis for devising a sensible, cost effective 
approach to contractor past performance. The FAR does give sonae recognition to 
market research in Part 1 1 for determining the availability of conunercial products in the 
marketplace for Government use. And the Non*Developmental Item (NDI) handbook 
does address market investigations for NDI purposes. But neither encompasses 
contractor past performance considerations, nor is it undertaken on a continuing basis. 



Process Integration & Teamwork 

This criterion deals with provisions for handling the integration of past performance 
considerations with the other factors and analyses that may be pertinent to a particular 
business area, or to die acquisition program or programs that constitute the business 
area. In addition, the criterion encompasses the teamwork and coordination needed for 
dealing with the cross-functional interests and perspectives that may be a factor in the 
larger more complex business areas. 

The process integration and teamwork provisions in the Contractor Evaluation Program 
are primarily addressed in the component that deals with the development of a business 
area strategy. This activity also contributes to the process by which the vertical, overall 
acquisition strategy is developed for a program area. 

At the present time FAR / DFARS model does not specifically address the integration 
nor the teamwork aspects for dealing with contractor past performance, either in the 
collection of information on contractor performance or in the use of this information for 
contractor selection purposes. Whereas, guidance in this area could be developed and 
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provided in a separate document, there does not appear to be recognition of the need for 
an integrated and cross-functionai teaming approach to the implementation of the 
contractor past performance policy. 



Value to the User 

This criterion deals with the capability to focus on the needs of the ultimate user and to 
provide past performance information that has value to the users - that is, government 
officials involved in the acquisition of products and services, including the solicitation 
of sources, the evaluation of offerors, and the award of contracts. 

The Contractor Evaluation Program is designed to ensure that it will provide useful 
information to users by incorporating the following features: 

• Users define the speci fic evaluation criteria to be used . This feature is based on one of the 
underlying principles embedded in the Contractor Evaluation Program; namely, use dictates 
collection. The specific 2q)proach for handling the past perfonnance contractors is developed at 
the business area level and part of this process provides for developing the specific criteria that 
will be used to evabiate the performance of contractor. The primary user of this information is 
the same organisation that collects the information, or that oversee its collection. 

• Users maintain local files on contracto r past performance . When past performance information 
is gathered on a continuing basis for future use by a particular business area, it will typically 
consist of either performance appraisal information or perfonnance tracking information. 
Performance appraisals are generated locally by government ofiBcials with contract management 
oversight responsibility. And cracking information is typically gathered from separate databases 
that cover quality and delivery performance. In either case, this information is filed locally and 
continuously updated for future use in the selection, of contractors to perform similar work. 

No provisions are made for the DFARS model to ensure that the needs of the ultimate 
users of the information will be considered in the selection of the evaluation criteria. 

Past Performance Information Sharing 

This criterion deals with the capability to share past performance information among 
government organizations. 

In the Contractor Evaluation Program, provisions are made for sharing two types of 
information within the business are-one type includes administrative information, and 
the other type includes the specific past performance information used by the other 
government organizations. This additional information could be appended to a central 
contractor registry or shnilar centralized system, through a lead site within the business 
area, or it could be provided separately. A brief description of each is provided in the 
following two paragraphs. 



• The administrative information includes on-line access to the full range of data from the Federal 
Procurement Data System (FPDS) for any product or service code of interest These data 
include the identity of contractors that provide various products and services to the government, 
including contract numbers, types, and dollar value. Additional information would also be 
available to include a synopsis of contract work statements; an indication if past performance 
information was available for a particular contractor at a certain location; the availability of 
planning information for the particular business area; and contact points for the purpose of 
obtaining additional information and coordinating with the otiier government organizations. 

• The past performance information would include information available at other business area 
sites, based on criteria used by the business area. This information would be accessible by direa 
contaa witij the otiier organizations by whatever means is established by the organization that 
maintains die information (e.g., telephone, e-mail, FAX, and database access). Provisions would 
ensure that access is provided only to authorized users. 



Section iV. Conclusions 



r^^T? "T"^ "''^ ^ P"^- '^^ ^""^ to provide our responses 

to two fundamental questions DUSD (AR) needs to consider to guide past performance 
pohqy xmplementation. In the second part are general lessons learned that we believe 

should gmde past performance policy implementation. The third part presents specific 
associated conclusions. 



A. Responses to DUSO (AR) Questions 

Question: Should DOD use past performance? 

Answer Yes, because: 
; • it makes good business sense 

• it is required by law and regulation 

• it be tailored to fit specific circumstances, although it is not clear who should 
do the tailoring and to what extent. 

Question: What information should be collected-what type of approach should be 
used and what du^tion and guidance should be provided? 

Answer: The DOD approach should follow these general principles: 

• Decentralized-The range of products and services, and the variance in the size 
scope, type, and complexity of contracts makes a standard, DOD-wide system ' 
impractical. Government and industry experience support a decentralized 
approach supported by general guidelines, decision rules, best practices, and 
information technology support. 

. Focused on Business Areas-The implementation of past performance should 
focus on mdividual business areas at the operating level that encompass similar 
products or services for which a coherent and congruous strategy can be 

developed by organizations with procurement authority and technical 
responsibility. 
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• Total Program Context-Past performance needs to be viewed in the context of a 
total program that goes beyond the collection and use of past performance 
information, and covers: 

• Analysis of individual business areas, to include both internal and 
external factors 

• Development of a sensible strategy for contractor past performance at the 
business area level. 

• Processes designed to implement the strategy for business areas in which 
the organization is active 

• Horizontally Intcgrated-The business area concept starts at the local level, 
where it is integrated with the overall acquisition strategy and procurement 
planning for the business areas. As business area alliances arc formed, it exerts a 
DOD-wide horizontal integration effect by joining similar business areas across 
the Services and DLA. The implementing direction needs to emphasize the need 
for this integration and coordination. 

• : User-Driven— The users of past performance information need to have the 

principal role in defining what information to collect, when to collect it, and how 
to make it available for their use in selecting contractors. And the users should 
include the technical, management, and procurement officials who arc involved 
/ in and responsible for naaking contractor selection decisions. 

• Share Information— Systems and processes for sharing past performance 
information among organizations depend on all of the above and should be dealt 
with after all of the above are dealt with. 

• Simple-To be effective, the past performance approach has to be easy to 

• understand and explain, without being simplistic, or it runs the risk of being 
misunderstood, ignored, or both. 



These general principles should guide the implementation of the following specific 
conclusions: 

• Past performance policy implementation should follow the tenets, procedures, 
and techniques of the Contractor Evaluation Program or a similar program. 

• The past performance information collection requirements of FAR Part 42 
should be implemented for commodity acquisitions, except for commercial 
products. 

• The past performance implementation requirements of FAR Part 42 should be 
tested on a pilot/prototype basis for the acquisition of services. 
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• The past performance informative requirements of FAR Part 42 should not be 
required for major/small systems. A pilot/prototype system should be tested for 
major/small systems, with emphasis on the evaluation of processes. 



The government and industry experience we reviewed was appUed to the issue of 
implementing FAR Part 42. 

Government Experience: There is a very Umited amount of Government experience 
avaUable to contribute to addressing this question. The past performance information 
systems we found in DOD acquisition accounted for a very smaU percentage of DOD 
actions or doUars and therefore are not a valid statistical or analytical bases for this 
question. However, the commodity-based systems we observed were successfully 
meeting their intended purposes. 



Based on the limited information avaUable, it would appear reasonable for DOD to 
implement FAR Part 42 collection requirements for commodity-type acquisitions. 
There is very Uttle data for Service-type acquisitions other than A&E and construction. 
Although the Corps of Engineers systems appeared to be successful for their highly- 
tailored appUcation, we do not feel comfortable extrapolating that specialized 
experience for all different types of Service procurements. 



For major systems, CPARS was an example of a successful system. The issue in this 
area is one of the benefit of coUecting vast amounts of past performance information 
over a six- to ten-year period when it may not be used for a similar contractor selection 
until ten to twelve years after first starting the collection. For example, the F-22 
development information is fi-om 5 years ago. It may be applicable to JAST. but that 
won't be for another 5 years. 

Industry Experience: There is an extensive amount of industry supplier evaluation 
program experience (the industry parallel to our defmition of a government past 
performance information system), albeit in commodity-type products, that supports the 
implementation of FAR Part 42 guidance by DOD. However, there is very litde 
industry experience with services and major/small system acquisitions using supplier 
evaluation program techniques. There is also an extensive amount of industry 
experience on approval of processes in addition to measuring contract performance. 



The benefits of establishing ongoing performance measurement, certification, and/or 
approval programs support implementing FAR Part 42 requirements. These benefits 
include: 

• Improved service 

• Decreased costs (trasportation, product/part, transaction (labor), payment terms 
inventory, operations) 

• Increased quality 

• Improved development time and introducton of new technologies 

• Increased customer satisfaction and loyalty 

• Higher employee morale 



The process essentially leverages the information that would otherwise be coUected for 
source selections by usuig it for performance measurement/feedback, certification, 
and/or approval. 



Question: How should infonnation be collected-Single system, DecentraUzed 
systems. Ad hoc only? 



Answer 
Single System: 

A single DOD-wide past performance information system would be effective only as a 
red and/or blue flag" system and could not provide the detailed analyses to support 
best-value, world-class contractor selection. 

The weight accorded past performance information in contractor selections has been 
considerably increased. Compared to what is envisioned in the new policies, contractor 
past performance evaluation generally has been conducted in a very circumscribed 
manner: 

• Past performance was a minor factor in source selection 

• Typically valued at 5-15% 

• Generally not a discriminator in the selection process. 

• Past perfomrance information coUection generally was ad hoc and concurrent 
with the source selection process 

• Forms/Calls to program managers and contracting officers on on-gomg and 
expired contracts 
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• Contractor submissions in response to RFPs, RFQs 
. Contract management performa.K:= measurement data was not designed to be 
used in source selection 

another considerable issue. 

With few excepdons. the accuracy of the '^Z'^X'^^ 
rXlTL is needed for any P-< P«— ^™ & that is used in 

r^coS*a=a^;::^:^7r~--"-- 

data. 




Legacy Systems Data 



With a few exceptions legacy system qualrtyand 
deliver data may not be of the quality needed to 
support past performance as a significant factor in 
selecting best-value, wortd-dass contractois. 



Low 



Data 
Quality 



f High 



July 1995 



January 2000 
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Decentralized System: 

DecntraUzed systems which are organized on business area basis are the most cost- 
e^^Z iprZch to UnpUmen^g FAR Par, 42 cottecion regutrentents. 

The reasons ttat led us to develop the Contractor Evaluation Program^ model are the 
SL^ suZt, this conclusion A decentralized system with a bustness ar^ f«us 

r «aevant detailed data to support best-value, world-class contractor 
S:t^oS X'^y wiU be irS^roved. but requires detailed procedures to 
insure quality data is provided to selection officials. 

Ad hoc Approach: 

This approach may not provide the systematic, accurate, relevant data that is needed to 
support past performance as a major factor in contractor selection. 

Systematic coUection of past performance data wiU help improve its useftilness in ^ 
L^^r s^Sn However, due to the distributed nature of thc^k^^that xs collected 
ZZT^ZZ. of source selection it is highly unlikely that it wdl be entuely 
relevSt, aSu^ or detaUed enough to support the evaluation weight bemg accorded to 
pir^rfo^'in co^^^ selections. Tlus ad hoc approach is charactensuc of the 
^r i^^ch past performance data was coUected in the past Improvements bemg 
Z^^y^DODtfother Fed^ Agencies in coUection «es - a^^p m ^e r^^ 
diction However until procedures for contractors bemg able to validate the data are 
adTo is^ruU ^^^^ imagine the ad hoc approach as being My capable of supporting 
past performance-based contractor selection decisions. . 
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B. Lessons Learned 

The following lessons learned are provided for DUSD (AR) consideration: 
. Acheiving full implementation can take 5 to 7 years. Nonetheless, the 

perspective and its evolving development offers vision, a road niap, and 

confirmation of direction for DOD. 



Bast ValtM/ Worid Class 




JUy 199S 



January 2000 



Linking past performance strategies to overall acquisiiton reform strategy and 
initiatives is critical. 

Partnering and multi-functional teamwork at all levels witii internal customers 
and suppliers are essential. 

All stakeholders must be identified and explicitly considered in process 
improvements. 

Information systems and accurate data are critical to implementation. 

The challenge to improve the supplier base is difficult, but can be achieved by 

working witii suppliers in a win-win relationship. 



C. Associated Conclusions 

There are a number of barriers to mdespread adoption of past performance as a major 
selection factor in a best-value context. 

• Low bidder mindset/culture 

• Risk avoidance culture 
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Uck of experience with subjective decisionmaking 

• Need to educate buyers thai there is a choice 

• Time to validate performance information 

• Quality and delivery data processes are weak/inaccurate 

• Lack of tools to collect accurate data 

• Impacts acquisition streamlining efforts to reduce PALT 

• Impacts productivity 

• Administrative burden 

Innovative change management training programs will be required to meet past 

^o^ce policy impLmenm^^^ 

objectives. 

. Cultural change to support other-than-"low-cost" mentality is slow to take place 
without new learning, team environment, management commitment, and sound 
automation systems. 

. Industry suppUer evaluation, performance measurement, and recognition of 
successes and techniques may need to be introduced. 

• .Government initiatives may need to be expanded and emphasized. 



Work with DOD contractors to develop: 

• Common awareness of DOD business past performance vision/strategy 

• Shared understanding of current reality/leverage points 

. Align actions for redesigning processes and implementing resource, technology, 
and organizational features 

• Collaborative review of progress throughout the cycle 



Emphasize long-term contracts. 

• Provide contractors with the confidence to do necessary long-term planning ar 
increase commitment. 

. Provide contractors with tangible evidence that you are serious about 
partnership. 

. Reduce cost by lengthening the period that contractors have to recover capital 
investments. 

• Reduce administrative costs of annual contrart award. 



On-going process evaluation programs should be considered. DCMCs Risk 
Assessment Model and PROCAS programs and planned initiatives like JACG-CPARS 
Supplier Assessment are similar to industry programs. 

. DCMC uses RAM and PRCX^AS programs evalute key contractors to improve 
their processes. 

. Designed for contract manageerant not to support source selection 

• Do not apply to most DOD suppliers 
. JACG-CPARS focus on contractor capabUity to perform future contracts based 

on performance risk and assessmscnt of key processes. 
Process evaluation adds an important dimension to judging future performance. 
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